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SUMMARY 


PURPOSE 

The  purpose  of  this  study  is  to  examine  the  utility  of  simulation  for  training  at  the 
individual,  unit,  and  joint  force  levels  of  readiness  and  to  propose  guidelines  for  the 
development  of  new  technology  relevant  to  training,  particularly  in  the  area  of  advanced 
distributed  simulation. 

BACKGROUND 

The  use  of  simulation  for  training  at  all  levels,  from  firing  a  rifle  in  a  school  to  a 
large-scale,  joint  exercise  in  the  field,  is  a  widely  accepted  practice.  Simulations  are  used 
routinely  in  such  cases  as  the  initial  familiarization  of  novices  in  the  operation  and 
maintenance  of  simple  equipment  (e.g.,  radio  or  compass),  or  complex  equipment  (e.g., 
aircraft  or  nuclear  control  station),  or  in  handling  potentially  catastrophic  events  (e.g., 
failure  of  an  engine  in  an  aircraft,  or  an  over-heating  boiler  or  a  fire  on  a  ship).  Questions 
about  the  utility  of  simulation  can  (and  should)  arise  as  to  the  ability  of  a  simulator  to 
represent  the  operation  of  the  actual  equipment  in  its  real  world  environment  or  of  a  combat 
model  to  represent  actual  combat  (i.e.,  fidelity  of  the  simulation);  the  specific  tasks  for 
which  a  simulator  can  offer  effective  training  (e.g.,  whole  or  part  training,  maintenance  or 
operation);  the  ability  to  use  skills  learned  in  a  simulator  on  the  actual  equipment  in  its  real- 
world  environment  (called  "transfer");  the  relative  costs  and  effectiveness  of  using 
simulators  for  training  as  compared  to  those  of  using  the  actual  equipment  (simulators  tend 
to  be  less  expensive);  and  the  optimum  combination  of  the  use  of  simulators  and  actual 
equipment  for  most  effective  training  at  least  overall  cost  (this  should  be  the  bottom  line). 
In  addition,  military  personnel  should  be  concerned  when  use  of  a  simulator  is  advocated  to 
save  costs  and  funds  for  OPTEMPO  (operating  tempo  for  ships,  aircraft,  and  vehicles), 
that  is,  training  and  exercises  with  actual  combat  equipment  are  to  be  cut 

Up  to  about  15  years  ago,  the  term  "simulation"  referred  almost  entirely  to  devices 
used  for  training  individuals  to  perform  some  of  the  tasks  associated  with  their  jobs  as 
pilots,  navigators,  maintainers,  repairers,  firemen,  communicators,  and  the  like.  Today, 
advances  in  high-speed,  wide  bandwidth  communication  networks  and  high-performance 
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computers  have  made  possible  the  use  of  distributed  interactive  simulation  for  collective 
training  at  all  levels,  including  joint  training  and  readiness.  This  new  capability  supports 
large-scale  combat  training  exercises,  both  at  Service  and  joint  levels,  precisely  at  a  time 
when  funds  for  field  exercises  are  being  reduced  because  of  cost  and  environmental 
factors.  It  is  important  to  point  out  that  the  development  of  distributed  interactive 
simulation  for  training  will  also  improve  the  contributions  that  this  technology  can  make  to 
component  and  joint  readiness,  test  and  evaluation,  mission  preparation  and  rehearsal,  and 
the  development  and  evaluation  of  tactics  and  doctrine. 

In  the  treatment  that  follows,  we  consider  four  types  of  simulation  used  for  training: 
stand-alone  simulators,  networked  interactive  simulation  (virtual  simulation),  exercises  on 
instrumented  ranges  (live  simulation)  and  computer-based  combat  models  (constructive 
simulation).  We  consider  two  types  of  training — individual  and  collective  training, 
including  joint  training — and  two  places  where  military  training  occurs — institutions 
(schools)  and  operating  units.  Data  on  the  effectiveness  and  cost  of  training  by  type  of 
simulation,  and  by  place  of  training  vary  in  their  completeness  and  quality. 

FINDINGS 

This  paper  considers  issues  with  respect  to  research  and  development,  procurement 
and  utilization  that  are  central  to  the  development  of  policy  concerning  the  use  of  simulation 
for  military  training.  Findings  concern  budgets  related  to  simulation  and  training,  the  cost 
and  effectiveness  of  simulation,  experience  of  the  Services  with  simulation,  distributed 
interactive  simulation,  and  the  technologies  relevant  to  simulation  and  training. 

Budgets  Related  to  Simulation  and  Training 

Budget  data  on  the  costs  of  simulation  and  training  are  not  reported  regularly  in  the 
Department  of  Defense.  This  means  that  our  ability  to  discern  trends  in  costs  of  different 
types  of  training  (e.g.,  individual  or  collective  training),  or  to  compare  the  costs  of  training 
at  a  school  to  training  in  an  operational  unit,  or  to  direct  research  and  development  towards 
areas  of  highest  pay-off  for  training  effectiveness  and  cost  is  seriously  limited.  The  costs 
of  OPTEMPO  are  not  well  known;  in  fact,  we  found  two  different  estimates  of  the  costs  of 
OPTEMPO  that  differ  by  a  factor  of  2.3.  In  this  report,  data  from  various  sources  are 
compiled  to  estimate  what  some  of  these  costs  are;  some  of  the  data  are  highly  reliable, 
while  some  lack  clear  definition  as  to  what  cost  elements  may  or  may  not  be  included.  Our 
findings  are  summarized  in  Figure  S-1  (and  are  discussed  more  fully  in  Chapter  II). 
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Type  of  Expenditure _ 


RDT&E 

Simulators  for  weapon  systems 
Technology 

Training  equipment 
Training  methods 
Modeling  and  simulation 
Joint  commands 
DMSO 
ARPA 


Procurement  of  simulators 
Aviation  $  0.800 

Non-system  devices  0.243 


Q,0$Z 


Amount^ 

Period 

Source 

$0,336 

FY91-97 

Frost  &  Sullivan  (1 993)2 

0.101 

FY94 

MATRIS  (1993)3 

0.038 

FY94 

MATRIS  (1993)3 

0.019 

FY  91-92 

IG  (1993)4 

0.073 

FY  93-94 

DMSO  (1994)5 

0.103 

FY  92-97 

ARPA  (1993)6 

FY  91-97 

Frost  &  Sullivan  (1993) 

1.100 

NA 

NA 

14.4 

FY94 

MMTR  (1993)7 

NA 

NA 

12.7 

9.4 

LMI  (1993)8 

LMI  (1993) 

21.4 

FY91 

Angier  et  al.  (1992)® 

0.425 

FY94 

Briefing  material 

0.369 

FY  91-97 

Frost  &  Sullivan  (1993) 

All  others 

Models  and  simulations 
Military  Construction 


Individual  training  in  institutions 
Individual  training  in  units 
Collective  training  in  insthutbns 
Collective  training  in  units 
OPTEMPO 

Joint  exercises 
Simulator  maintenance 


^  Amount,  in  billions,  for  year  cited  or  average  of  years  noted. 

^  Frost  &  Sullivan  (1 993) 

3  Manpower  and  Training  Research  Information  System  (1993) 

*  Inspector  General  (1993) 

^  Defense  Modeling  and  Simulation  Office  (1 994) 

^  Advanced  Research  Projects  Agency  (1993) 

^  Military  Manpower  Training  Report,  FY  94  (1993) 

^  Logistics  Management  Institute  (1993) 

^  Angier,  Alluisi,  and  Horowitz  (1992) 

NA  Not  Available 

Figure  S-1.  Estimated  Annuai  Expenditures  for  Training  and  Simulation  in 
Terms  of  RDT&E,  Initial  investment,  and  Operating  and  Support  of  Training 
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Individual  training  occurs  at  schools  and  cost  $14.4  billion  in  FY  94.  Collective 
training  occurs  in  operational  units  but  its  costs  are  not  reported  regularly;  it  was  estimated 
to  cost  $12.7  billion  in  FY  93.  Two  estimates  were  found  for  expenditures  for  OPTEMPO 
(fuel,  consumables,  repairs  and  maintenance  for  flying  hours,  steaming  days,  and  vehicle 
miles):  $9.4  billion  (FY  1993,  LMI)  and  $21.4  billion  (FY  1991,  Angier,  Alluisi  and 
Horowitz).  The  larger  estimate  includes  costs  for  repair  and  depot  maintenance  not 
included  in  the  smaller  estimate;  except  for  this  item,  we  were  not  able  to  resolve  the  basis 
for  these  widely  different  estimates.  The  fact  that  two  FFRDCs  developed  such  different 
estimates  for  the  cost  of  OPTEMPO  illustrates  the  importance  of  having  reliable  and  regular 
estimates  of  the  costs  associated  with  various  aspects  of  simulation  and  training. 

The  procurement  of  simulators  for  training  costs  about  $1.1  billion  per  year 
(average  of  FY  91-97);  RDT&E  and  support  of  simulators  cost  an  additional  $0.5  and  $0.4 
billion,  respectively.  The  most  expensive  simulators  are  for  aviation;  they  cost  about  $0.8 
billion  per  year  to  procure  (73  percent  of  the  costs  of  simulators  for  all  types  of  weapon 
systems);  non-system  training  devices  cost  $0.2  billion  (22  percent);  simulators  for  all 
other  types  of  weapons  cost  $0.6  billion  per  year  (5  percent). 

Expenditures  for  RDT&E  on  simulators  and  training  equipment  average  about  $0.4 
billion  per  year,  training  methods  about  $0.04  billion,  and  modeling  and  simulation  (for 
DMSO,  ARPA,  and  the  Joint  commands)  about  $0. 19  billion  per  year. 

Significant  issues  in  the  cost  of  training  are  who  is  trained  and  where  training 
occurs.  "Who  is  trained"  refers  to  individual  is  opposed  to  collective  training;  "where 
training  occurs"  refers  to  institutions  (i.e.,  schools)  or  operational  units.  The  problem  in 
ascertaining  the  cost  of  each  type  of  training  at  various  places  (and  the  related  issue  of 
evaluating  their  cost  and  effectiveness)  arises  because,  except  for  individual  training  at 
institutions,  there  is  no  regular  or  consistent  report  that  identifies  the  cost  of  all  other  types 
of  training.  Our  estimates  of  what  these  costs  are  were  compiled  from  a  variety  of  sources, 
as  shown  in  Figure  S-2.  The  costs  of  RDT&E  and  initial  investment  are  not  included  in 
this  figure;  we  were  not  able  to  find  any  usable  data  on  the  costs  of  collective  training  in 
institutions  or  of  individual  training  in  units.  We  estimate  individual  training  in  institutions 
cost  $14.4  billion  in  FY  94;  this  is  a  reliable  figure.  Other  costs,  as  estimated  by  a  variety 
of  sources,  are  as  follows:  collective  training  in  units  cost  $12.7  billion  in  FY  1993; 
OPTEMPO  cost  either  $9.4  or  $21.4  billion  (FY  91),  according  to  two  different  estimates; 
and  Joint  exercises  cost  about  $0,425  billion  (FY  1994). 
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Where  Training  Occun 


Institution 

Unit 

Who  is 

trained 

Army 

Navy 

Marine  Corps 

Air  Force 

Bvtvoe,  FY  94'' 

Not  available: 

Individual  (on-the-job)  training  in 
units  for  those  trained  only  on  the 
job,  job  familiarization  for  those 
trained  at  schools,  and  training  for 
those  assigned  jobs  that  do  not 
match  their  MOS  ® 

Individual 

Recruit 

Officer  acquisition 
Specialized  skill 

Flight 

Professional  development 
OSUT  (Army)  ^ 

$1.2  B 
.5 

4.1 

2.2 
.9 
.3 

14.4 

Unit  training  $12.7  FY  93^ 

Collective 

Army  Combat  Training 
Centers,  FY  95 

.5  B® 

OPTEMPO  9.4  FY  93^ 

21.4  FY913 

Other  data  not  available 

Joint  exercises  0.425  FY  94^ 

1  MMTR  (1993). 

2LM1  (1993). 

3  Angier,  Alluisi,  and  Horowitz  (1992). 

4  Briefing  Estimate. 

5  AUSA  (1994):  about  one-third  of  this  total  is  for  transport  to  and  from  the  training  center  (Fig.  II-  ) 

6  CBO  (1994)  estimates  6-10  percent  mismatch  between  job  and  MOS  in  the  Army,  FY  1993  (p.  32). 

7  0SUT:  Army  One-Station  Unit  Training:  combines  Recruit  and  Specialized  skill  training. 


Figure  S-2.  Estimated  Annual  Expenditures  for  Training,  by  Type  of  Training 
and  Where  Training  Occurs  (does  not  include  RDT&E  and  initial  investment) 

Cost  and  Effectiveness  of  Simulation 

Evaluations  of  the  cost  and  effectiveness  of  simulation  for  training  are  based  almost 
exclusively  on  the  use  of  flight  and  maintenance  simulators  and  of  computer-based 
instruction  for  initial  individual  training  at  institutions.  The  cost  and  effectiveness  of 


simulation  for  more  advanced  individual  training  in  units  or  for  collective  training  at 
institutions  or  in  units  for  component  or  joint  training  has  not  received  much  attention.  The 
available  findings  show  that  simulators  are  cost-effective  for  initial  flight  and  maintenance 
training  in  institutions:  they  train  as  well  as  does  actual  equipment  and  cost  less  to  procure 
and  use.  This  finding  applies  also  to  computer-based  instruction  as  compared  to 
conventional  classroom  instruction.  Simulators  are  a  good  investment.  The  cost  of  their 
procurement  can  be  amortized  in  periods  of  one  to  four  years.  However,  optimum 
combinations  for  the  use  of  simulators  and  actual  equipment  for  various  types  of  training 
have  not  been  studied;  nor  have  such  critical  issues  as  the  rates  of  learning  and  forgetting, 
which  are  basic  to  determining  how  much  and  when  simulators  or  actual  equipment  are  best 
used  for  initial  and  refresher  training.  The  decision  to  use  new  simulators  on  the  basis  of 
equal  effectiveness  and  less  cost  than  actual  equipment,  the  rule  used  at  present,  overlooks 
the  fact  that,  for  military  purposes,  one  should  seek  simulators  that  provide  increased 
performance  effectiveness  at  the  same  or  lesser  cost. 

Service  Experience  with  Simulation 

Information  was  collected  on  over  50  simulations,  some  considered  by  the  four 
military  services  to  be  successful  and  some  to  have  problems.  Features  associated  with 
successful  simulations  include:  high-user  acceptance;  timely  availability  of  a  well- 
developed  training  plan  to  show  how  the  simulator  should  be  used;  genuine  contribution  of 
the  simulator  to  training  demonstrated  by  features  for  performance  measurement  and 
feedback;  acceptable  cost;  and  minimum  interference  by  simulators  with  existing  norms  for 
training  with  actual  equipment  (i.e.,  little  reduction  in  budgets  for  flying  hours,  vehicle 
miles,  and  steaming  hours).  Problems  with  existing  simulations  are  attributed  to 
inadequate  or  nonexistent  training  plans,  discrepancies  between  the  performance  of 
simulators  and  actual  equipment,  and  the  absence  of  features  considered  to  be  important  for 
training,  e.g.,  motion  platforms,  sensors  (IR,  radar,  EW),  feedback  capability.  Almost  all 
of  these  limitations  are  the  result  of  decisions,  for  cost  or  other  reasons,  to  procure 
simulations  with  limited  or  no  capability  for  training  on  certain  tasks.  Current  technology 
appears  adequate  to  deal  with  most  of  the  deficiencies  reported,  provided  there  is  an  interest 
in  and  funds  are  appropriated  to  upgrade  and  improve  current  equipment 

The  following  trends  are  observable  with  respect  to  the  role  of  simulators  for 
training;  (1)  a  reduction  of  flying  hours,  up  to  50  percent  per  year  for  transport  aircraft, 
and  of  vehicle  miles  per  year  for  armor,  to  pay  for  the  cost  of  procuring  a  new  distributed 
training  system  for  close  combat;  (2)  use  of  simulators  to  complement  flight  training  for 
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advanced  combat  aircraft,  with  a  slight  reduction  in  flying  hours;  and  (3)  the  development 
of  low-cost,  unit-training  devices  to  be  placed  at  most  flight  bases  for  use  as  modifiable, 
part-task  trainers  for  electronic  warfare,  target  recognition,  and  digital  terrain  land  mass 
training  Current  flight  simulators  will  be  phased  out  because  they  are  too  expensive  to  be 
placed  at  all  flight  bases.  A  significant  trend  is  the  development  and  use  of  advanced 
distributed  simulators  in  all  Services  for  collective,  combat  training  but  it  is  too  soon  to 
assess  their  utility  for  training. 

Distributed  Interactive  Simulation 

Distributed  Interactive  Simulation  (DIS)  systems  now  being  developed  have  a  great 
potential  for  improving  unit  and  joint  training  and  joint  readiness  (and  will  also  have 
important  applications  in  other  areas,  such  as  test  and  evaluation,  tactics  and  doctrine,  and 
mission  rehearsal).  However,  the  development  of  training  strategies  (i.e.,  ways  to  use  DIS 
systems  to  improve  collective  training)  is  lagging  behind  development  of  the  underlying 
technology  in  hardware  and  software,  communications,  and  standards  needed  to  support 
DIS.  A  problem  that  will  surely  arise  is  how  best  to  use  these  systems  for  training  (and  test 
and  evaluation)  and  how  to  evaluate  their  effectiveness  and  cost.  Meaningful  tests  will 
require  large  numbers  of  people  to  serve  as  test  subjects  over  extended  periods  of  time,  as 
well  as  development  and  testing  of  scenarios  and  performance-measuring  techniques. 
Personnel  must  also  be  trained  to  design  and  conduct  tests  and  collect  reliable  test  data 
needed  to  evaluate  the  utility  of  DIS  for  training  and  to  support  a  decision  to  procure  the 
required  equipment 

Technologies  Relevant  to  Simulation  and  Training 

A  review  of  technologies  critical  to  simulation  and  training  included  R&D  on 
distributed  interactive  simulation  in  areas  concerned  with  networks,  semi-automated  forces, 
terrain  and  environment,  range  instrumentation,  individual  combatants,  virtual  environ¬ 
ments,  and  training  technology.  Areas  that  receive  major  funding  ($1(X)  million  a  year  or 
more)  are  networks,  terrain  and  environment,  and  range  instrumentation.  Within  these 
areas,  increased  attention  should  be  given  to  develop  the  Asynchronous  Transfer  Mode 
(ATM)  protocol  for  DIS,  support  for  exercise  management,  compatibility  between  data 
bases  for  terrain  and  environment,  radio-frequency  network  bandwidths  for  range 
instrumentation,  and  communication  standards.  Less  support  is  being  given  to  methods  of 
training,  design,  and  use  of  individual  combatants  in  DIS  and  SAPOR.  There  is  a  need  for 
development  of  methods  of  evaluating  performance  in  all  areas,  clarification  of  appropriate 
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applications  of  individual  combatants  (including  dual-use  potential)  in  DIS,  and  effective 
ways  to  use  and  measure  the  training  potential  of  most  DIS  systems. 

RECOMMENDATIONS 

Priorities  for  Research  and  Development 

Review  the  research  and  development  programs  on  simulation  and  training  to 
assure  that  they  focus  on  areas  of  highest  expenditures  and  potential  payoff.  These  are 
aviation  simulators,  aviation  training,  OPTEMPO,  joint  training  and  readiness,  and 
distributed  interactive  simulation. 

Distributed  Interactive  Simulation 

Extend  efforts  to  evaluate  the  cost  and  effectiveness  of  training  technology  beyond 
the  limited  areas  of  flight  and  maintenance  simulators  and  computer-based  instruction. 
Attention  should  be  given  to  methods  of  evaluating  the  cost  and  effectiveness  of  distributed 
interactive  simulation  systems  such  as  the  Close  Combat  Tactical  Trainer  (CCTT), 
Synthetic  Theater  of  War  (STOW),  and  the  Multi-Service  Training  Testbed  (MDT2),  and  to 
the  use  of  modeling  for  joint  training,  where  the  major  issues  are  likely  to  be  validity  and 
effectiveness. 

Training  Strategies 

Give  high  priority  to  the  development  and  evaluation  of  training  strategies  for  the 
use  of  distributed  interactive  simulation  for  large-scale  Service  training  and  joint  training  for 
readiness.  Emphasize  efforts  concerned  with  estimating  performance  in  joint  training 
exercises  because  this  will  provide  a  way  to  estimate  joint  training  readiness.  Development 
and  evaluation  of  methods  of  measuring  performance  in  joint  training  should  be  started 
soon  in  order  to  be  in  place  when  needed  to  evaluate  new  DIS  systems,  as  they  become 
available  in  three  to  five  years. 

Performance  Data  Base 

Develop  a  data  base  system  that  provides  a  systematic  way  to  compile  performance 
data  that  become  available  from  large-scale  Service  and  joint  exercises.  Evaluating  the 
effectiveness  of  new  DIS  systems  for  training  (e.g.,  CATT,  MDT2,  and  STOW)  will  be 
difficult  because  of  the  large  resources  needed  for  test  subjects  (military  personnel  for 
extended  exercises),  data  collection,  and  personnel  qualified  to  conduct  tests  and  analyze 
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results.  A  comprehensive  data  base  can  provide  a  means  to  determine  lessons  learned  from 
many  different  tests  to  supplement  the  large-scale  test  programs. 

How  Much  Training  is  Enough 

Support  research  and  development  on  key  issues  of  how  much  training  is  enough 
and  how  often  refresher  training  must  occur  to  maintain  joint  training  readiness.  There  is  a 
significant  absence  of  critical  information  on  training  with  respect  to  learning  and  forgetting 
curves  and  on  developing  optimum  combinations  of  the  use  of  simulators  and  actual 
equipment  for  various  applications.  This  must  be  remedied  in  order  to  maximize  the 
benefits  available,  from  investments  in  different  equipment  and  methods  of  training. 

Combat  Models 

Insufficient  attention  has  been  paid  to  the  use  and  effectiveness  of  models  used  for 
joint  and  large-scale  Service  training.  A  program  to  evaluate  the  verification,  validity,  and 
accreditation  of  models  used  in  unit,  battle  command  staff,  and  joint  training  should  be 
undertaken. 

Cost-EHectiveness  Paradigm 

The  utility  of  simulations  has  generally  been  decided  on  the  basis  of  equal 
effectiveness  and  less  cost  than  the  use  of  actual  equipment.  Although  this  is  an  acceptable 
guideline,  attention  should  be  directed  towards  the  development  of  simulations  that  increase 
performance  effectiveness  at  no  appreciable  increase  in  cost  or  at  lesser  cost.  Military 
effectiveness  benefits  from  training  for  improved  performance  rather  than  merely  for 
current  levels  of  proficiency. 

Cost  Data  on  Training 

Except  for  data  furnished  to  Congress  on  individual  training  in  institutions,  there  is 
a  notable  absence  of  regularly  reported  data  on  the  costs  of  collective  training  in  institutions 
and  operational  units,  on  the  costs  of  OPTEMPO,  exercises,  on-the-job  training,  joint 
training  and  on  the  acquisition  of  training-related  hardware  and  software.  Undertake  the 
development  of  cost-reporting  systems  that  will  identify,  define,  and  make  regularly 
available  cost  data  on  training  that  is  needed  to  support  policy  decisions  in  each  of  these 
areas. 
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On-the*Job  Training 

Only  limited  attention  has  been  given  to  the  cost  and  effectiveness  of  on-the-job 
training,  in  comparison  to  formal  training  in  institutions.  Review  current  R&D  activities  in 
order  to  plan  a  more  vigorous  R&D  effort  toward  on-the-job  training.  New  developments 
in  computer-based  instruction,  distance  learning,  and  portable,  miniaturized  electronic  job 
aids  make  this  an  attractive  area  for  improving  the  effectiveness  and  reducing  the  costs 
(largely  hidden)  of  on-the-job  training. 


I.  PURPOSE  AND  BACKGROUND 


PURPOSE 

The  purpose  of  this  study  is  to  examine  the  utility  of  simulation  for  training  at  the 
individual,  unit,  and  Joint  force  levels  of  readiness  and  to  provide  guidelines  for  the 
development  of  new  technology  relevant  to  training,  particularly  in  the  area  of  advanced 
distributed  simulation. 

BACKGROUND 

"'Simulation'  has  always  been  a  difficult  issue,  tricky  to  analyze  because  it  is 
surrounded  by  a  semantic  quagmire,  and  obscured  by  a  miasma  of  emotion,  over-claims, 
and  flawed  analyses,  unilluminated  by  dependable  statistics  on  costs  or  effectiveness."  So 
writes  a  friend  who  reviewed  an  earlier  version  of  this  report.  This  report  cannot  eliminate 
the  miasma,  least  of  all  that  which  is  self-generated,  but  it  can  try  to  find  a  path  through  the 
quagmire.  Let  us  start  with  the  issue  of  readiness,  the  potential  contribution  of  simulation 
to  readiness,  and  the  definition  of  some  key  terms. 

My  highest  priority  for  the  Department  is  to  keep  our  forces  ready  to  fight. 

Defense  Planning  Guidance,  Secretary  of  Defense, 

September  28, 1993 

The  men  and  women  who  serve  under  the  American  Flag  will  be  the  best 

trained,  best  equipped,  best  prepared  fighting  force  in  the  world,  so  long  as 

I  am  President. 

President  Bill  Clinton, 
February  1993 

Everything  is  simulation  except  combat 

Defense  Science  Board,  1993 

Winning  the  cold  war  in  a  bi-polar  world  did  not  it  turns  out  lead  to  general  peace 
in  our  multi-polar  world.  Reductions  in  the  defense  budget  and  in  force  structure,  together 
with  changes  in  the  external  threat  increase  the  difficulty  of  sustaining  military  readiness  to 
deal  with  less  predictable  adversaries  in  less  obvious  places.  The  components  of  readiness 
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include  training,  materiel,  personnel,  supply,  and  infrastructure.  Although  the  focus  of  this 
paper  is  on  simulation's  role  in  training  for  Joint  readiness,  we  believe  that  advanced 
distributed  simulation  has  utility  not  only  for  training,  but  for  the  other  components  of 
readiness  as  well.  The  reason  is  that  military  exercises  conducted  in  synthetic  theaters  of 
war  provide  an  opportunity  not  only  for  training  but  also  for  evaluating  concepts  and 
procedures  for  weapon  systems,  supply,  tactics,  and  doctrine — whether  these  be  "conven¬ 
tional"  or  innovative.  An  indirect  but  potentially  very  powerful  contribution  that  simulation 
can  bring  to  defense  planning  and  management  is  that  the  Department's  disparate 
communities,  such  as  those  concerned  with  research  and  development,  operations,  acquisi¬ 
tion  and  the  budget,  may  begin  to  identify  and  use  common  methods,  assumptions  and 
findings  in  developing  policy,  strategy,  budgets,  and  tactics. 

We  are  more  dependent  on  simulation  than  is  generally  acknowledged.  New 
weapons,  such  as  aircraft,  tanks,  and  ships,  are  now  designed  and  developed  with  a  major 
reliance  on  simulation;  this  reduces  the  need  for  mock-ups,  eliminates  some  tests  of 
hardware,  and  shortens  redesign  of  components  to  assure  interoperability  on  the  battlefield. 
Of  course,  confirmational  system  testing  remains  necessary,  but  now  it  can  occur  towards 
the  end  of  the  development  cycle.  Decisions  to  build  the  major  components,  often  at  great 
cost,  can  be  made  with  far  less  risk.  Battle  plans  and,  to  some  extent,  actual  tactics  can  be 
based  more  reliably  on  extensive  simulation  with  combat  models.  The  Defense  Science 
Board  (DSB)  (1993)  in  its  recent  report  on  simulation,  readiness,  and  prototyping,  said  that 
"Everything  is  simulation  except  combat."  The  DSB  examined  the  potential  use  of 
simulation  for  many  purposes  of  concern  to  defense,  such  as  training,  test  and  evaluation, 
mission  rehearsal,  and  system  acquisition.  It  is  clear  that  simulation  will  have  much 
widespread  utility,  but  consideration  of  applications  outside  training  is  beyond  the  scope  of 
this  effort. 

It  is  unthinkable  that  training  to  cope  with  potentially  catastrophic  events,  such  as 
the  failure  of  a  nuclear  reactor,  or  of  an  engine  on  an  Apollo  lift-off,  or  of  the  first  attempt 
to  land  on  the  Moon,  could  be  conducted  in  any  way  other  than  by  the  use  of  simulators. 
Some  widely  accepted  examples  of  successful  training  that  depended  on  simulations  are 
shown  in  Figure  I-l. 
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Landing  on  the  Moon 

Actual  training  not  possible  on  Earth 

Top  Gun  Fighter  Weapons 
School 

Improved  Navy  exchange  ratio  in  air  combat 
over  North  Vietnam  from  2.4  to  12.5 

Commercial  Airlines 

FAA:  Simulator  training  alone  qualifies  a  pilot 
to  fly  a  new  airplane  for  the  first  time  on  a 
revenue  flight 

National  Training  Center 

OPFOR  experience  at  NTC  (simulation)  is  a 
decided  advantage 

Nuclear  power  plants 

No  significant  accidents  (after  Three  Mile 

Island)  in  military  and  civilian  operations 

Canadian  Armor  Trophy 

Extensive  training  with  SIMNET  and  UCOFT 
he  Iped  Army  win  CAT  for  the  first  time  in  1 987 

73  Easting 

Participants  report  that  success  in  Gulf  Wbr 
battle  was  based  on  tactical  experience  gained 
in  SIMNET,  NTC  and  Grafenwoehr 
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Figure  1-1.  Examples  of  Successful  Training  with  Simulation 

It  should  be  noted,  however,  that  such  reliance  on  simulation  is  an  artifact  of  our 
generation.  When,  in  World  War  I  and  again  in  World  War  II,  the  armed  forces  of  the 
United  States  were  called  upon  to  redress  abject  military  unpreparedness,  they  were 
required  to  simulate,  to  conduct  training  with  wooden  mockups  of  weapons,  lacking  as 
they  did  the  real  ordnance.  Ever  since,  for  military  professionals,  "simulation"  has  been  a 
loaded  word,  a  term  applied  invidiously  to  any  training  involving  a  simulacrum  of  military 
equipment,  or  admitting  pretended  movement,  shooting,  or  communications.  Even  today, 
in  virtually  any  military  headquarters,  stentorian  proclamations  can  be  heard  to  the  effect 
that  "this  command  does  not  simulate!  It  trains  by  actually  doing  everything  it  must  do  in 
combat,  with  real  people,  real  equipment,  and  real  weapons!" 

Military  reservations  about  simulation  revolve  around  the  contemporary  issue  of 
OPTEMPO  (operating  tempo  reported  as  flying  hours  per  month,  vehicle  miles,  or  sailing 
days  per  year).  OPTEMPO  occasions  much  of  the  cost  of  maintaining  military  forces  in  a 
time  of  nominal  peace.  Despite  its  high  cost,  it  offers  a  major  opportunity  to  prepare  for 
war  by  training  and  practice  with  real  weapons  and  vehicles.  No  pilot,  sailor,  or  tanker 


joined  the  Service  to  become  proficient  in  combat  skills  by  "sitting  in  a  box"  and  practicing 
with  a  simulation. 

The  budgeting  conventions  of  the  Department  of  Defense  and  the  Congress  have 
almost  always  purchased  simulators  by  reducing  OPTEMPO,  seeking  both  to  reduce 
overall  expenditures  and  to  offset  the  cost  of  procurement.  The  word  "simulation"  then 
bears  the  burden  of  two  generations  of  short-sighted  Service  programming,  during  which 
the  Office  of  the  Secretary  of  Defense  (OSD)  and  the  Congress  were  taught  that  a 
simulator,  especially  an  expensive  aviation  simulator,  should  be  funded  only  when  test  data 
could  be  produced  to  demonstrate  that  the  device  would  "pay  for  itself"  by  being 
demonstrably  cheaper  than  actual  equipment  for  training  and  at  least  as  effective. 

Recent  advances  in  simulation  technology  patently  require  that  the  Department  of 
Defense  and  the  armed  Services  lay  aside  the  baggage  surrounding  the  terminology,  and 
look  anew  at  opportunities  presented.  A  useful  way  to  begin  would  be  with  definitions; 
these  are  presented  below. 

The  present  leaders  of  the  Department  of  Defense  perceive  "simulation"  as  a  major 
policy  instrument,  recourse  to  which  can  enable  the  Services  to  thrust  ahead  buoyed  by  the 
strong,  commercially  driven  technologies  of  the  Information  Age.^  The  Secretary  of 
Defense  recently  remarked  that  "taking  units  to  the  field,  aircraft  to  the  air,  or  ships  to  sea  is 
no  longer  the  only  way  we  can  do  effective  training.  Simulation  is  another.  And  the 
combination  of  field  tests  and  simulation  is  probably  the  most  effective  way  to  train,  and 
we're  just  beginning  to  introduce  simulation. 

During  the  past  twenty  years,  the  U.S.  military  services  have  employed  three  forms 
of  simulation  to  portray,  for  training  or  any  other  purpose,  what  happens  when  one  military 
force  engages  another:  (1)  constructive  simulations,  mathematical  models  of  combat,  from 
duels  between  weapons  to  wars  among  nations;  (2)  live  (real  or  subsistent)  simulations, 
involving  engagements  among  actual  military  forces  and  vehicles  with  simulated  weapon 
effects,  and  (3)  virtual  (apparently  real)  simulations,  comprising  interactions  among 
manned  simulators  of  weapon  platforms,  operating  in  wholly  synthetic,  computer- 


^  William  J.  Perry,  Secretary  of  Defense,  Remarks  to  the  Global  Air  and  Space  1994  International 
Forum,  5  May  1994. 

2  Ibid. 
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generated  environments.^  While  constructive  and  live  simulations  have  been  improved 
slowly  over  the  past  century  or  more,  virtual  simulation  is  a  recently  actualized,  dramatic, 
new  capability  based  on  modem  computers  and  displays.  Further,  only  in  the  past  decade 
has  inter-computer  programming  and  communications  made  it  possible  to  obtain  coherent 
outcomes  from  simulation  using  all  three  forms  of  simulation  interactively.  Hence,  the 
current  thrast  in  development  of  advanced  simulation  is  to  enable  a  "synthetic  theater  of 
war"  (STOW)  using  distributed  interactive  simulation  (DIS)  for  Joint  (inter- Service)  and 
combined  (inter-allied)  training. 

Military  training  occurs  in  one  of  four  distinctive  environments,  depending  upon 
whether  it  is  training  for  individuals  or  collectives,  and  upon  whether  it  takes  place  in  an 
institutionally  stmctured  environment,  or  within  the  resources  of  a  military  unit.'^  Thus,  in 
considering  how,  where,  and  when  to  introduce  simulation  for  training,  the  following 
paradigm  is  helpful: 

Who  is  trained 


Individual 


Collective 


Where  training  occurs 
Institution  Unit 


The  Services  have  found  it  easiest  to  introduce  simulation  for  training  and  to 
demonstrate  its  cost-effectiveness  in  Block  1,  individual  training  in  an  institution.  This  is 
so  because  (1)  simulation  is  most  efficiently  used  within  a  time-constrained  curriculum  by 
an  institutional  staff  that  can  assure  continuous  usage  of  expensive  simulators,  computer- 


3  P.  F.  Gorman,  "The  Future  of  Tactical  Engagement  Simulation,"  Proceedings  of  the  1991  Summer 
Computer  Simulation  Conference,  The  Society  for  Computer  Simulation  (SCS),  Place,  D.,  ed., 

pp.  1181-1186. 

^  For  a  further  discussion,  see  P.  F.  Gorman,  "Training  Technology,"  in  Trevor  Dupuy,  editor-in-chief. 
International  Military  and  Defense  Encyclopedia,  Macmillan  Publishing  Company,  Riverside,  NJ, 
1993. 


based  battle  games  or  other  simulative  equipment,  and  (2)  because  reliable  data  on  training 
transfer  is  easier  to  collect  from  relatively  homogenous  populations  of  students  while  they 
are  captives  in  scheduled  courses. 

Conversely,  Services  procure  expensive  simulations  for  Block  2,  individual 
training  in  an  operational  unit,  at  the  hazard  of  subsequent  adverse  audits  by  the 
Government  Accounting  Office  showing  that  no  consistent,  effective  employment  of  the 
simulation  apparatus  had  occurred  amid  the  hurly-burly  of  a  unit's  routine  activities. 
Because  units  often  can  neither  assure  adequate  utilization  nor  reliable  records,  simulations 
are  typically  maintained  by  a  training  center  or  other  institution  located  nearby,  so  that 
usage  can  be  efficiently  scheduled,  and  spread  across  personnel  assigned  to  several  units. 
Examples  are  aviation  flight  simulators,  and  shoulder-fired  air  defense  weapon  simulators. 
As  the  price  of  computers  and  displays  drops,  risks  associated  with  introducing  simulation 
for  training  in  units  will  decline.  For  example,  the  Air  Force  is  developing  Unit  Training 
Devices  that  will  be  located  at  every  flight  base;  these  are  relatively  inexpensive  devices  that 
can  be  used  for  training  pilots  on  a  variety  of  tasks  (see  Chapter  IV). 

Block  3,  collective  training  in  an  institution,  is  an  area  that  has  undergone  major 
growth  over  the  past  two  decades,  and  usually  depends  upon  either  institutionally  directed 
wargaming,  a  constructive  simulation  (such  as  the  Army's  Battle  Command  Training 
Program)  or  upon  an  instrumented  range,  a  subsistent  simulation  using  a  network  of 
weapon  emulators  and  positioning  sensors  that  enable  a  synthetic  combat  environment. 
Examples  of  the  latter  are  "Strike  University"  (Fallon  Naval  Air  Station,  Nevada),  RED 
FLAG  (Nellis  Air  Force  Base,  Nevada),  and  the  Army's  Combat  Training  Centers  (Fort 
Irwin,  California;  Fort  Polk,  Louisiana;  and  Hohenfels,  Germany).  The  Services  also 
maintain  less  ambitious  instrumented  ranges  for  smaller  tactical  performances,  such  as  the 
Navy's  Air  Combat  Maneuvering  Instrumentation.  Introducing  advanced  technology 
enabling  joint  and  combined  exercises  on  such  ranges,  exploiting  all  three  forms  of 
simulation,  constitutes  a  major  opportunity  to  enhance  and  estimate  readiness. 

Block  4,  collective  training  in  an  operational  unit,  is  clearly  another  high-payoff 
area  for  readiness  but,  to  date,  relatively  little  investment  has  been  made  in  simulation 
designed  expressly  for  use  in  units,  by  unit  commanders,  training  their  forces  for  combat 
readiness.  The  Services  have  to  date  resisted  building  training  subsystems  into  their  battle 
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equipment.  Even  exceptions,  notably  MILES^-type  laser-weapon-emulators  or  ACMI^ 
pods,  involve  strap-on  devices.  Up  until  the  recent  past,  investments  in  simulation  for 
collective  training  at  institutions  (Block  3)  could  be  mote  readily  amortized  over  time  than 
buying  simulation  equipment  to  be  used  by  units.  But  as  digitization  of  the  battlefield 
proceeds,  as  dependence  on  digitally  transmitted  information  grows  throughout  the  armed 
forces,  opportunities  increase  to  build  into  unit  equipment  itself  the  means  to  generate  a 
synthetic  battle  environment  for  training  or  for  operational  rehearsal.  Hence,  collective 
training  in  units  (Block  4)  ought  to  be  perceived  as  a  promising  growth  area  for  simulation 
for  training. 

We  can  now  see  a  convergence  of  two  streams  of  events.  Reductions  in  the  funds 
available  for  OPTEMPO  (i.e.,  combat  training  in  the  field)  reductions  in  the  number  and 
size  of  ranges  available  for  such  training,  coupled  with  concern  for  noise  and 
environmental  factors,  all  combine  to  reduce  the  opportunities  for  live  simulation  and 
combat  training  with  actual  equipment  under  realistic  conditions.  However,  large  advances 
in  computer  capability  and  in  high-capacity  global  communications — ^much  of  it  the  result 
of  commercial  interest — ^have  facilitated  the  growth  of  constructive  and  virtual  simulation. 
The  current  development  of  distributed  interactive  simulation  (DIS),  or  geographically 
broadcast  virtual  simulation,  motivates  the  concern  of  this  paper  with  the  military  value  of 
simulation  for  training. 

DEFINITIONS 

For  sake  of  clarity,  we  define  and  discuss  briefly  some  of  the  terms  and  concepts 
used  throughout  this  report. 

Individual  and  Collective  Training 

Military  training  falls  into  two  classes  called  Individual  and  Collective  Training. 
Individual  training  provides  the  skills  needed  to  accomplish  particular  jobs,  associated  with 
about  500  Military  Occupational  Specialties.  These  skills  are  developed  by  almost  20,000 
courses  in  recruit,  officer  acquisition,  specialized  skill,  and  flight  training  and  by 
professional  development  education  for  officer  and  enlisted  personnel.  Collective  training 
refers  to  development  of  the  skills  needed,  in  addition  to  the  individual  ones,  for 


Multiple  Integrated  Laser  Engagement  System 
Air  Ctxnbat  Maneuvering  Instnimentaticxi 


5 

6 


1-7 


individuals  to  operate  as  crews  or  teams  in  aircraft  cockpits,  command  and  control  stations, 
ship's  bridges,  communication  centers,  tanks,  and  so  on. 

Institutional  and  Unit  Training 

Formal  individual  training  occurs  at  schools  (i.e.,  fixed  facilities  with  faculty  and 
administrative  support);  this  is  called  Institutional  Training.  Most  collective  and  some 
important  individual  training — on-the-job  or  experiential  learning — occurs  in  operational 
units  and  commands.  This  is  called  Unit  Training.  Although  most  units  will  have  a 
training  section,  both  the  trainers  and  trainees  must  accommodate  to  the  vaiying  demands 
placed  on  operational  units.  Unit  training  schedules  and  curricula  are  subject  to  constant 
change;  this  rarely  occurs,  except  by  design,  in  institutional  training.  As  noted  above, 
some  of  the  most  effective  institutional  training  for  collectives  is  subsistent  simulation  for 
units  exercising  under  a  faculty  at  an  instrumented  range. 

The  student  loads  and  costs  of  individual  training  at  institutions  are  reasonably  well 
known  because  they  are  reported  annuaUy  to  Congress  in  the  Military  Manpower  Training 
Report  This  cannot  be  said  of  any  other  type  of  training,  either  in  schools  or  in  military 
units.  Each  Service  is  responsible  for  unit  training,  and  for  providing  combat-ready 
components  to  the  U.S.  combatant  commands,  i.e.,  the  CINCs,  such  as  the  Commander(s) 
in  Chief,  U.S.  Atlantic  Command  (USACOM),  U.S.  Pacific  Command  (USPACOM),  and 
Europe  (US  AEUR).  Because  of  the  draw-down  in  force  structure,  joint  training  to  develop 
and  assess  joint  readiness  (i.e.,  preparation  for  the  ways  in  which  forces  of  the  several 
Services  will  operate  together)  will  now  receive  special  attention  by  the  Services,  Joint 
Staff,  and  the  Office  of  the  Secretary  of  Defense."^  A  major  significance  of  distributed 
interactive  simulation  is  the  contribution  it  may  make  to  opportunities  for  Joint  exercises, 
and  for  assessing  readiness  not  otherwise  available  or  too  costly,  as  well  as  for  the 
compilation  of  performance  data  on  how  well  joint  forces  can  accomplish  various  types  of 
missions  or  exercise  scenarios. 


^  Defen^Science  Board  and  Secretary  of  Defense;  Senior  Readiness  Oversight 

Council,  Deputy  Sectary  of  Defense;  Readiness  Working  Group.  Under  Secretary  of  Defense 
(  ersonnel  and  Readiness);  and  Joint  Requirements  Oversight  Council,  Ihe  Joint  Staff. 
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Simulation 


The  imitative  representation  of  one  system  or  process  by  means  of  the 
functioning  of  another. 


Webster's  Ninth  New  Collegiate  Dictionary, 
Merriam- Webster  Inc.,  Smithfield,  MA,  1987 

An  inferior  substitute  imitating  an  original. 

Roget's  II,  The  New  Thesaurus, 
Houghton  Mifflin  Company,  Boston,  M A,  1980 

In  principle,  a  simulation,  which  is  the  product  of  a  simulator,  is  a 
representation  of  a  system  (including  organizations)  by  another  system,  the 
"other  system"  being  the  simulation.  "Hie  first  system  need  not  be  real;  it 
too  may  be  a  simulation  (and  so  forth)  although  such  iteration  is  rarely 
required. 


James  T.  Westwood  in  Trevor  N.  Depuy,  editor-in-chief. 
International  Military  and  Defense  EncyclopetUa, 
MacMillan  Publishing  Company,  Riverside,  NJ,  1993 

The  term  simulation  refers  to  the  general  process  by  which  certain  real- 
world  systems,  operations  or  phenomena  are  initiated  using  representational 
devices  such  as  models,  game  boards,  computers  or  other  equipment.  The 
focus  of  the  term  is  on  the  process  of  imitating  (or  simulating)  the  reality  of 
concern,  rather  than  on  the  models  or  items  used  or  the  rules  to  be  followed. 

Frances  B.  Kapper  in  Trevor  N.  Depuy,  editor-in-chief. 
International  Military  and  D^ense  Encyclopedia, 
Macmillan  Publishing  Company,  Riverside,  NJ,  1993 


Simulator 

A  device  that  enables  the  operator  to  reproduce  or  represent  under  test 
conditions  phenomena  likely  to  occur  in  actual  performance. 

Webster's  Ninth  New  Collegiate  Dictionary, 
Merriam-Webster  Inc.,  Smithfield,  MA,  1987 

A  simulator  is  a  device  or  facility  that  produces  simulations. 

James  T.  Westwood  in  Trevor  N.  Dupuy,  editor-in-chief. 
International  Military  and  Defense  Encyclopedia, 
Macmillan  Publishing  Co.,  Riverside  New  Jersey,  1993 

Stand-Alone  Simulators 

Flight  simulators  to  train  pilots  were  built  almost  as  soon  as  aircraft  were  designed. 
Other  types  of  simulators  are  used  to  train  operators  and  maintenance  personnel  for  such 
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purposes  as  crew  cockpit  coordination,  in-flight  refueling,  identifying  malfunctions  in  radar 
and  sonar  systems,  and  controlling  the  diving  of  submarines.  All  of  these  simulators  are 
grouped  here  on  the  basis  that  they  are  self-contained  and  stand-alone  devices  that  are  used 
to  train  personnel  in  many  of  the  individual  and  crew  skills  required  to  operate  and  maintain 
weapon  and  support  systems.  The  Services  use  thousands  of  these  devices.  In  the  future, 
some  of  these  devices  may  be  modified  so  that  they  can  interact  with  other  simulators  in 
distributed  interactive  simulation  systems. 

Networked  Interactive  Simulators 

The  environment  that  is  produced  by  networks  of  interactive  simulators  used  for 
combat  training  is  called  "virtual  simulation"  and  SIMNET  (Simulator  Networking)  is  the 
best  known  prototype  of  this  class  of  large  scale  simulation.  Networked  simulators  are 
designed  to  be  used  for  unit  and  joint  training  in  large  scale,  two-sided  engagements  with 
various  types  of  weapon  systems  and  command  and  control  groups.  Individuals  must  have 
been  trained  to  operate  weapons,  e.g.,  tanks  and  aircraft,  before  they  can  receive  collective 
training  using  networked  simulators.  Future  improvements  in  the  physical  and  functional 
fidelity  of  networked  simulators,  as  well  as  reductions  in  their  cost,  may  permit  both 
individual  and  unit  training  to  take  place,  using  the  same  simulator.  Networked,  interactive 
simulations  now  under  development  include: 

BFTT  Battle  Force  Tactical  Trainer 

BFTT  Battle  Force  In-port  Training 

(XTT  Close  Combat  Tactical  Trainer 

AGPT  Ausbildungsegerat  Gefechtssimulator  Panzer  Truppe  (German  SIMNET) 

CATT  Combined  Arms  Tactical  Trainer  (British  SIMNET) 

MDT2  Multi-Service  Distributed  Training  Testbed 

STOW  Synthetic  Theater  of  War 

Instrumented  Ranges 

Instrumented  ranges  can  record  the  location  and  timing  of  events  that  take  place 
between  opposing  sides  in  simulated  combat  of  troops  using  actual  weapons  and  platforms, 
called  "subsistent  simulation"  or  "live  simulation."  These  ranges  provide  objective 
performance  data  on  both  sides  for  use  in  after-action  reviews  for  unit,  joint,  and  combined 
Service  training. 


Computer-Based  Combat  Models 

Models  may  be  either  iconic  (enlarged  or  miniature  versions  of  real  objects), 
symbolic  (abstract  versions,  words,  diagrams),  mathematical,  or  computer-based.  The  use 
of  computer-based  combat  models  in  simulation  is  called  constructive  simulation 
(Defense  Science  Board,  1993). 

The  catalogue  (12th  edition)  of  The  Joint  Staff  lists  about  1000  computer-based 
models  that  simulate  all  types  of  combat  and  combat  support,  and  that  are  employed  for 
many  purposes,  analytical  as  well  as  training,  in  the  Department  of  Defense.  Examples  of 
combat  models  are  Corps  Battle  Simulation  (CBS,  Army),  Janus  (Army),  Enhanced  Naval 
Wargaming  System  (ENWGS,  Navy),  Research,  Evaluation,  and  Systems  Analysis 
(RESA,  Navy),  Air  Warfare  Simulation  (AWSIM,  Air  Force),  and  Joint  Conflict  Model 
(JCM,  Joint  Staff).  The  widespread  use  of  combat  models  for  many  types  of  military 
analyses  is  a  measure  of  their  usefulness,  convenience,  and  popularity.  However,  their 
validity  has  been  strongly  challenged  (see  Davis  and  Blumenthal,  1991). 

Distributed  Interactive  Simulation 

The  concept  of  Distributed  Interactive  Simulation  (DIS),  also  called  Advanced 
Distributed  Simulation,  envisions  that  any  combination  of  stand-alone  simulators,  networks 
of  simulators,  live  exercises  on  instrumented  ranges  and  computer-based  combat  models 
may  be  employed  interactively  and  in  real  time.  A  good  example  is  the  Synthetic  Theater  of 
War  (STOW),  now  being  developed  by  the  Advanced  Research  Projects  Agency  (ARPA) 
with  multi-Service  participation  for  joint  combat  exercises.  This  type  of  simulation  will 
support  exercises  that  permit  interactions  in  real  time  between  actual  vehicles  and  weapons 
on  instrumented  ranges  (e.g.,  aircraft  at  Red  Flag  and  Top  Gun,  ground  vehicles  at  the 
National  Training  Center,  and  naval  vessels  off  San  Diego),  networked  simulators,  and 
computer-based  combat  models. 

Virtual  Reality 

This  term  is  given  to  a  class  of  simulators  that  produce  a  so-called  experience  of 
"total  immersion"  for  the  participants.  The  devices  that  are  used  to  produce  such  striking 
effects  include  helmet-mounted  displays  (to  produce  three-dimensional  visual  imagery  and 
auditory  effects),  instrumented  gloves,  force  feed-back  to  various  parts  of  the  body,  control 
devices  (in  addition  to  gloves),  motion  bases,  and  the  computers,  data  bases,  and 
algorithms  needed  to  create  a  semblance  of  generally  simplified  reality.  The  effects. 


although  still  cartoon-ish  in  nature,  can  be  impressive.  Systems  of  this  sort  have  great 
possibilities  for  remote  manipulation  and  micro-surgery  (even  at  a  distance,  with  visual  and 
force  feed-back)  in  the  real  world,  as  well  as  for  training  and  amusement. 

Virtual  reality  is  a  technology  that  is  now  undergoing  rapid  development  for  such 
varied  purposes  as  entertainment,  medical  education  and  surgery,  architectural  design  and 
tactical  planning.  Virtual  reality  is  not  considered  in  this  paper  except  for  some  technology 
to  support  interfacing  with  virtual  environments. 

On-the-Job  Training 

On-the-job  training  (OJT)  is  "Any  pre-planned  use  of  work  resources  (people, 
equipment,  facilities)  in  the  work  environment  primarily  for  the  purpose  of  training 
someone  to  produce  work."  (Carpenter-Huffman,  1980,  p.  2).  On-the-job  training  occurs 
in  operational  units  after  the  completion  of  technical  training  courses  in  schools.  Although 
all  training  in  units  might  be  called  on-the-job  training,  in  practice  OJT  refers  only  to  the 
real  world  training  of  individual  specialty  skills  still  needed  after  the  completion  of  school. 
For  some  types  of  skills,  training  may  occur  either  at  schools  (followed  by  OJT)  or  only  by 
on-the-job  training  (see  Chapter  H). 

Cost  and  Effectiveness 

Training  is  intended  to  develop  a  specified  level  of  proficiency  in  a  designated 
subject  matter  area,  such  as  communications,  radar  maintenance,  aircraft  navigation,  sonar 
operations,  or  data  processing.  Effectiveness  refers  to  a  measure  of  the  extent  to  which  one 
method  of  training  (e.g.,  classroom  instruction,  computer-based  instruction,  simulation, 
field  exercises)  succeeds  in  helping  trainees  acquire  the  skills  and  knowledge  specified  by 
the  standard  established  for  completion  of  a  course.  Cost  refers  to  the  expenses  incurred  in 
providing  training  and  may  include  some  or  all  of  the  costs  of  course  development, 
including  research,  test  and  evaluation  of  training  procedures  and  equipment,  acquisition  of 
the  hardware  and  software  needed  for  training,  and  operation  and  maintenance  (i.e.,  the 
delivery  of  training).  In  this  paper,  we  take  the  position  that  information  on  both  the 
effectiveness  and  cost  of  a  method  of  training,  compared  to  some  alternative  method,  is 
needed  in  order  to  determine  the  preferred  method  of  training. 


ADVANTAGES  AND  DISADVANTAGES  OF  SIMULATORS  AND 
SIMULATIONS 

We  conclude  the  introduction  to  this  report  and  a  description  of  its  background  by 
listing  many  of  the  advantages  and  disadvantages  often  found  in  the  literature  on 
simulation.  They  represent  conventional  wisdom  and  may  or  may  not  be  correct  when 
applied  to  particular  simulators  and  simulations  used  for  specific  purposes.  The  remainder 
of  this  report  attempts  to  supplement  such  wisdom  by  providing  and  discussing  data  and 
information  that  can  be  used  to  support  developments  in  technology  and  policy  intended  to 
improve  the  utility  of  simulation  for  training  and  joint  readiness,  wherever  it  may  be 
needed. 

Advantages 

•  Trains  many  tasks  as  well  as  would  the  use  of  actual  equipment. 

•  Costs  less  than  actual  equipment  to  procure  and  use. 

•  Provides  training  that  otherwise  requires  use  of  actual  equipment. 

•  Permits  training  for  dangerous  and  potentially  catastrophic  conditions  not 
otherwise  possible  in  actual  equipment 

•  Reduces  risk,  safety  hazards,  and  wear-and-tear  from  use  and  maintenance  of 
actual  equipment,  and  extends  useful  life  of  equipment  needed  for  combat. 

•  Reduces  impact  that  actual  equipment  has  on  environment,  and  the  noise  and 
use  of  ranges  have  on  communities. 

•  Protects  operational  security  of  tactics  and  mission  rehearsal,  of  electronic 
warfare,  and  sensitive  performance  characteristics  of  new  weapons. 

•  Provides  features  needed  for  instruction  and  feedback,  including  performance 
measures. 

•  Saves  on  use  of  fuel,  munitions,  support  facilities. 

•  Provides  initial  familiarization,  trains  procedures  (less  so  for  advanced 
students),  and  avoids  risk  to  actual  equipment  by  novices. 

•  Permits  training  not  readily  affordable  with  actual  equipment. 

•  Permits  use  of  actual  or  anticipated  threat  conditions  to  examine  doctrine,  effect 
of  the  combat  environment,  and  potential  responses. 

•  Permits  testing  of  effectiveness  of  plans  and  tactics  prior  to  actual  operations. 

•  Permits  joint  training  of  DoD  forces  and  of  coalition  forces. 

•  Can  recreate  significant  battles  to  identify  key  events  and  to  train  leaders. 
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Disadvantages 

•  Could  adversely  affect  training  readiness  if  use  of  simulators  reduces 
OPTEMPO,  as  well  as  confidence  in  use  of  actual  equipment. 

•  Uninformed  by  data  from  actual  performances  of  weapon  systems  and  service 
crews,  virtual  and  constructive  simulations  can  be  dubiously  valid. 

•  Require  funds  to  procure  and  use. 

•  May  instill  habits  that  are  incorrect  for  use  in  actual  equipment. 

•  Limited  fidelity  of  visual  resolution,  sensors,  aerodynamics,  and  motion 
platforms  can  lead  to  inadequate  or  misleading  training. 

•  Differing  response  characteristics  of  visual  displays  and  of  motion  platforms 
often  leads  to  motion  sickness. 

•  There  is  a  potential  compromise  of  security  if  sensitive  tactics  and  equipment 
features  are  gained  by  access  to  networks  that  support  interactive  simulators. 

•  Accurate  performance  of  weapons  and  platforms  in  simulations  requires 
accurate  engineering  models  and  data  that  may  not  be  available  or  are  expensive 
to  develop  in  such  areas  as  aerodynamics  and  hydrodynamics,  lethality  of 
weapons,  vulnerability,  weather,  visibility,  target  detection  with  various 
sensors,  terrain. 

Unresolved  Issues  in  Simulation 

•  Identification  of  types  of  training  that  cannot  be  accomplished  using  distributed 
interactive  simulation  and  development  of  viable  alternatives. 

•  Validation,  verification,  and  accreditation  of  high-level  models  to  be  used  in 
estimating  Joint  readiness  and  training  status. 

•  Organizational,  cultural,  and  management  issues  that  must  be  considered  if 
distributed  interactive  simulation  is  implemented  on  a  grand  scale. 

•  •  Optimum  combinations  for  use  of  simulation  and  actual  weapons  and  platforms 
in  exercises  to  achieve  maximum  performance  effectiveness  at  least  cost  for 
most  applications. 


II.  BUDGETS  RELATED  TO  SIMULATION  AND  TRAINING 


In  this  chapter,  we  try  to  identify  the  costs  of  various  components  of  simulation  and 
training,  such  as  the  procurement  of  simulators  and  simulations,  training  at  schools  and  in 
the  operational  commands,  and  research  on  methods  of  training.  To  the  extent  possible, 
we  try  to  identify  trends  in  the  budgets  for  these  activities  over  recent  years.  We  also 
present  this  data  in  a  common  format  that  provides  estimates  of  annual  expenditures  in 
these  areas. 

Except  for  individual  training  at  institutions,  there  is  no  single  source  that  identifies 
all  of  the  funds  that  are  allocated  to  simulation  and  training  each  year.  This  is  due  largely  to 
the  fact  that  funds  that  support  individual  training  at  institutions  come  from  different 
appropriations  than  those  for  other  types  and  places  of  training;  in  addition,  funds  for  unit 
training  come  from  appropriations  that  also  support  operation  and  maintenance,  base 
support,  and  OPTEMPO.  An  attempt  to  estimate  the  costs  of  training  from  diverse 
accounts  is  likely  to  be  incomplete  and  probably  lack  standard  definitions  as  to  what  the 
various  cost  elements  relevant  to  training  should  or  should  not  include.  The  actual  costs  of 
on-the-job  training  in  units  are  virtually  hidden  and  are  very  difficult  to  identify.  To  the 
best  of  our  ability,  we  will  point  out  whatever  qualifications  may  apply  to  the  cost  data  we 
present.  Nevertheless,  we  believe  that,  despite  its  imprecision,  the  information  presented 
here  is  the  most  comprehensive  compilation  of  costs  associated  with  simulation  and  training 
that  is  now  available. 

The  costs  of  individual  training  at  institutions  are  known  because  they  are  reported 
annually  to  Congress  in  the  Military  Manpower  Training  Report  (MMTR).  The  MMTR 
describes  the  costs  and  amounts  of  individual  training  at  institutions  for  recruit,  skill,  initial 
officer,  and  flight  training  and  professional  development  education.  This  report  does  not, 
for  example,  provide  information  on  the  costs  or  amounts  of  on-the-job  technical  training 
that  takes  place  in  the  operational  commands  to  which  personnel  are  assigned  after 
completing  initial  individual  training;  these  costs,  if  they  were  knowable,  are  funded  by  the 
Operation  and  Maintenance  (O&M)  and  other  accounts.  Collective  training  occurs  mainly, 
but  not  exclusively,  in  operational  units  and  is  supported  by  funds  in  the  O&M  accounts 
that  also  provide  for  base  operations,  supplies  and  maintenance.  There  is  no  assurance. 


except  as  may  be  determined  by  a  subsequent  audit  of  actual  expenditures,  that  funds 
allocated  to  collective  training  were  actually  spent  for  that  purpose;  such  audits  are  not 
conducted  regularly.  Local  commanders  have,  and  should  have,  discretionary  authority  to 
re-distribute  these  funds  as  dictated  by  local  circumstances.  It  is  not  unheard  of,  therefore, 
that  funds  intended  for  training  are  actually  spent  to  repair  broken  water  mains  or  to  provide 
supplies  for  which  no  other  funds  are  available.  Funds  for  the  procurement  of  simulators 
are  sometimes  (but  not  imiformly)  included  in  the  costs  of  the  weapon  and  support  systems 
for  which  they  provide  training.  Funds  for  the  development  and  procurement  of  wargames 
and  combat  models  by  the  Unified  and  Specified  Commands  are  reported  here  only  because 
they  became  available  as  the  result  of  a  special  inquiry  by  the  Inspector  General.  Other 
comments  on  our  cost  data  are  made  below.  Sources  of  the  data  that  are  reported  here  are 
identified  in  the  figures  and  text  that  follow. 

Procurement 

Figure  11- 1  shows  total  projected  DoD  outlays  for  simulation-related  activities  by 
the  Services  for  the  seven  years  FY  1991-97,  in  constant  FY  1992  dollars.  This 
information  is  derived  from  Department  of  Defense  data  files  on  prime  contract  awards  for 
Training  Aids  and  Devices,  according  to  Federal  Stock  Classification  6900,  and  on 
program  summaries  of  planned  procurements  made  available  by  the  Services;  it  is  collected 
by  a  commercial  organization  that  prepares  market  surveys  and  is,  therefore,  not  an 
authoritative  document  The  total  amount  over  this  period  is  $13.5  billion  for  an  average  of 
about  $1.9  billion  per  year.  These  funds  provide  for  the  purchase  and  maintenance  of 
simulators,  as  well  as  for  research,  development,  and  test  and  evaluation  of  simulators  and 
training  devices.  The  amounts  spent  by  each  Service  are  similar  but  fluctuate  from  year  to 
year  as  a  result  of  a  large  procurement  related  to  the  introduction  of  a  major  weapon  system 
by  a  particular  service.  Marine  Corps  outlays  are  included  in  the  Navy  figure;  "Other"  is 
primarily  the  Defense  Logistics  Agency. 

The  same  data,  in  Figure  II-2,  show  that  the  procurement  of  simulators  and  services 
needed  for  their  support  account  for  over  80  percent  of  the  total  outlays  related  to 
simulations;  RDT&E  accounts  for  about  18  percent 
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Figure  IM.  Total  DoD  Outlays  for  Simulation-Related  Activities  by  Service, 
FY  91-97  (Source:  Frost  &  Sullivan,  1993,  p.  1-8.) 


Figure  11-2.  Total  DoD  Outlays  for  Simulation-Related  Activities  by  Functional 
Area,  FY  91-97  (Source:  Frost  &  Sullivan,  1993,  p.  1-5) 
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Figure  II-3  shows  DoD  outlays  for  the  procurement  of  simulators  by  type  of 
weapon  and  support  system.  A  brief  description  of  the  categories  follows: 

Aviation  Includes  specialized  aircraft-related  training  devices,  in 

addition  to  conventional  flight  simulators.  It  also  includes 
training  devices  embedded  in  or  carried  on  aircraft  for  in¬ 
flight  training  of  pilots  and  non-pilots  (navigators, 
engineers).  It  does  not  include  procurement  of  training 
aircraft  or  of  pilot  training  services  that  utilize  only  aircraft. 

Non-System  Training  Devices  Includes  training  devices  of  various  generalized  types  not 

identified  specifically  with  a  particular  system,  e.g., 
devices  for  electronics  and  hydraulics  training  and 
computer-based  instruction  devices.  (May  include  training 
devices  for  small  systems.) 

Missiles  All  types  of  training  devices  associated  with  missiles. 

Electronic  Warfare  Intelligence  collection  and  countermeasures  training 

devices  not  specified  as  onboard  ships,  aircraft, 
submarines,  or  armored  vehicles. 

Maintenance  System-specific  devices  for  training  military  personnel  to 

repair  aircraft,  ships,  and  other  platforms  and  the  equip¬ 
ment  associated  with  them. 

Battlefield  Armor  Command  All  battlefield  or  soldier-related  training  equipment. 

Training  devices  for  tanks,  fighting  vehicles,  and  other 
armored  weapons  systems.  Training  devices  for 
command,  control,  and  communications  systems  (C3); 
tactical  and  strategic  wargames;  and  other  leadership 
training. 

Surface  ships  All  training  devices  and  equipment  associated  with  ships 

and  surface  warfare. 

Submarines  All  training  devices  and  equipment  associated  with 

submarines,  submarine  warfare,  sonar  and  antisubmarine 
warfare. 

The  procurement  of  simulators  for  aviation  systems  accounts  for  about  $5.6  billion, 
or  72  percent  of  funds  spent  for  simulators  for  all  types  of  weapon  systems  ($7.8  billion 
for  FY  1991-1997).  This  is  followed  by  $1.7  billion  (23  percent)  for  Non-System 
Training  Devices.  The  procurement  of  simulators  for  all  other  weapon  systems  accounts 
for  $0.4  billion  (5  percent). 
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Figure  11-3.  Total  DoD  Procurement  for  Simulation  by  Weapon  and  Support 
Systems,  FY  91—97  (Source:  Frost  &  Sullivan,  1993,  p.  1—8) 


Modeling  and  Simulation 

Annual  expenditures  for  modeling  and  simulation  in  support  of  training  in  the 
Unified  and  Specified  Commands  for  1991  and  1992  are  shown  in  Figures  11-4  and  II-5. 
Estimates  of  the  costs  of  modeling  and  simulation  in  the  Unified,  Specified,  and  the 
Combined  Forces  Commands  were  collected  from  the  fund  managers  at  the  Defense 
Modeling  and  Simulation  Office  and  the  Joint  Staff,  J-7  and  J-8,  by  the  Inspector  General 
of  the  Department  of  Defense  (Inspector  General,  1993). 

Distinguishing  between  funds  explicitly  earmarked  for  training  models  md 
those  earmarked  for  other  models  comes  down  to  some  very  subjective 
choices.  Despite  the  subjectivity  of  the  data,  however,  the  two  graphs 
effectively  compare  general  funding  patterns  across  the  commands.  As  the 
different  scales  between  the  two  graphs  imply,  the  CINCs  received  greater 
funding  support  in  1991  (about  $25  million)  than  they  did  in  1992  (about 
$12.5  million)  from  the  three  funds.  (Inspector  General,  1993,  p.  24) 

Similar  data  on  modeling  in  the  individual  services  are  not  available. 
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Figure  li-4.  Annual  Expenditures  for  Modeling  and  Simulation  in  Support  of 
Training  in  the  Unified  and  Specified  Commands,  1991 
(Source:  Inspector  General,  1993) 


Figure  11-5.  Annual  Expenditures  for  Modeling  and  Simulation  in  Support  of 
Training  in  the  Unified  and  Specified  Commands,  1992  (Source:  Inspector 

General,  1993)  # 
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The  [amount  of]  funds  available  through  the  Joint  Staff  have  been  fairly 
consistent:  about  $8  to  $10  million  through  the  Joint  Modeling  and 
Simulation  Support  Program  and  about  $25  million  Arough  the  CINC 
Initiatives  Fund.  The  first  two  years  of  Defense  Modeling  and  Simulation 
Initiative  funds  have  hovered  around  $30  to  $40  million. 

Inspector  General,  1993,  p.  24. 

Unlike  the  Defense  Modeling  and  Simulation  Initiative  funds,  not  all  of  the 
Joint  Staff  monies  for  the  two  funds  discussed  are  spent  directly  on 
modeling  and  simulation  support  for  training.  The  principal  purpose  of  the 
J-8  funds  is  to  develop  and  maintain  analytical  capability  within  the  Unified 
and  Specified  Commands.  The  primary  purpose  of  the  J-7  funds  is  to 
support  unanticipated,  emergent  needs  of  the  Unified  and  Specified 
commands.  For  example,  a  review  of  the  1992  expenditures  under  the 
CINC  Initiatives  fund,  reveals  about  $2  million,  or  about  10  percent,  spent 
directly  on  modeling  and  simulation  support  for  the  CINCs.  During  the 
same  period,  the  Joint  Modeling  and  Simulation  Support  Program  spent 
about  $5  million,  or  50  to  65  percent  on  software  and  hardware  to  support 
modeling  and  simulation. 

Inspector  General,  1993,  p.  24 


Individual  Training 

Funds  allocated  to  the  training  of  individuals  and  the  amount  of  individual  training 
provided  in  man-years  at  schools  are  shown  in  Figure  II-6.  The  training  load,  in  man- 
years,  combines  the  time  spent  by  all  students  in  all  courses,  whether  they  last  days,  or 
years.  In  FY  1994,  about  174  thousand  man-years  of  training  were  provided  at  a  cost  of 
about  $14.4  billion;  this  is  a  reduction  of  7.4  thousand  (4  percent)  in  man-years  and 
$1.2  billion  (8  percent)  in  funds  from  FY  1993.^  When  looked  at  as  the  number  of  people 
who  aire  trained,  this  accounts  for  about  1.2  million  graduates  of  over  20,000  courses  in 
FY  1994.  In  terms  of  funding,  the  Army  spends  about  20  percent  more  than  the  Navy  for 
individual  training  in  FY  1994  and  almost  70  percent  more  than  the  Air  Force.  It  is 
intriguing  to  note  that  the  ratio  of  support  to  individual  training  is  about  0.8  for  man-years 
and  0.3  for  funds  (FY  94). 

Figure  11-7  shows  the  training  loads  (man-years  of  training)  and  funding  for  each 
category  of  individual  training,  such  as  recruit  training,  skill  training,  and  flight  training. 
The  largest  amount  of  funds  for  FY  1994  is  allocated  to  Specialized  Skill  Training 


1  For  FY  1995,  the  requested  training  load  is  180,000  man-years  (an  inaease  of  3.5  percent  over  FY 
1994)  and  funding  of  $14.0  billion  (a  decrease  of  2.9  percent  from  FY  1994).  (MMTR,  1995) 
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Figure  11-6.  Individual  Training  Loads  and  Funding  by  Service,  FY  93  and  FY  94 
(Source:  Military  Manpower  Training  Report,  FY  1994,  1993) 


Figure  11-7.  Individual  Training  Loads  and  Funding  by  Type  of  Training,  FY  93 
and  FY  94  [Source:  Miiitary  Manpower  Training  Report,  FY  1994,  1993] 


($4.1  billion),  followed  by  FUght  Training  ($2.2  billion).  When  we  divide  these  funds  by 
the  number  of  man-years  of  training  each  provides,  we  find  that  Specialized  Skill  Training 
costs  about  $42  thousand  per  man-year  while  Flight  Training  costs  about  $539  thousand 
per  man-year.  Further,  initial  flight  training  lasts  about  one  year  while  specialized  skill 
training  lasts  one-half  year  or  less.  Clearly,  flight  training  is  by  far  the  highest  cost  per 
person  type  of  individual  training  in  the  Department  of  Defense.  Note  that  the  funds  shown 
for  the  various  types  of  training  ($9.2  billion)  do  not  include  a  proportional  allocation  of 
overhead  funds  ($4.2  billion  for  direct  training  support,  training  base  support, 
headquarters);  or  of  Reserve  pay  and  allowances  ($1.0  billion).  If  these  were  included,  the 
costs  of  the  various  types  of  training  would  be  57  percent  larger  than  those  shown  in  the 
table,  although  the  total  would  not  change. 

Unit  Training 

Much  less  is  known  about  the  cost  of  individual  and  collective  training  in  units  and 
of  OPTEMPO  than  about  individual  training  at  schools.  When  individual  training  at  school 
is  completed,  additional  individual  (on-the-job)  and  collective  training  is  accomplished  in 
the  operational  units  to  which  individuals  are  assigned.  Operational  training  develops  the 
collective  skills  needed  to  perform  well  in  combat;  this  includes  crew  and  unit  training  at  all 
levels  within  the  Services  and  in  combined  and  Joint  exercises.  Unit  training  is  supported 
by  funds  in  the  Operation  and  Maintenance  accounts  that  also  provide  for  base  operations, 
security,  and  maintenance  of  base  facilities.  There  are  no  regular  reports  on  the  amount  of 
funds  allocated  to  on-the-job  training,  unit  training,  and  OPTEMPO;  there  are  a  few  ad  hoc 
estimates  of  these  costs. 

In  work  on  unit  training  performed  for  the  Office  of  the  Assistant  Deputy  of 
Defense  (Personnel  and  Readiness),  the  Logistics  Management  Institute  (LMI)  examined  all 
Program  Elements  (PE)  in  the  Operation  and  Maintenance  (0«&M)  funds  in  the  President's 
budget  submission  to  Congress  for  FY  1993.  The  total  O&M  budget  for  FY  93  was 
$67.2  billion.  Estimates  were  made  of  how  much  each  PE  contributes  to  unit  collective 
training,  in  accord  with  these  definitions; 

Operational  Units  -  These  are  the  units  that  are  being  trained.  They  are  ships, 
aircraft  squadrons,  divisions,  special  warfare  groups,  etc. 

All  O&M  is  Training  -  These  are  special  organizations  with  the  primary  mission  of 
providing  unit  collective  training  that  are  not  included  in  Major  Force  Program  8. 


Much  Support  -  These  are  organizations  that  we  feel  are  essential  to  unit  collective 
training,  but  which  may  have  significant  other  missions.  This  includes  such  things 
as  operational  headquarters  and  tactical  support  units  in  this  category. 

Some  Support  -  These  units  are  deemed  to  provide  some,  but  less  support  to  unit 
collective  training.  These  include  such  units  as  management  headquarters. 

No  Support  -  These  are  PEs  that  are  not  believed  to  contribute  to  unit  collective 
training.  This  should  be  not  construed  to  be  an  indicator  of  their  importance  or 
contribution  to  combat  forces  operations.  We  have  included  such  things  as  PEs 
specifically  related  to  the  acquisition  and  support  of  new  equipment  in  this  category. 

Unknown  -  These  are  PEs  with  classified  or  missing  PE  descriptions. 

Logistics  Management  Institute,  1993,  p.  1 

We  estimated  the  portion  of  each  operational  unit  PE’s  O&M  that  is  related  to  unit 
collective  training. 

In  the  case  of  Army  PEs,  we  were  able  to  identify  the  O&M  funding  that  Army 
considers  unit  collective  training  for  every  PE  through  the  training  MDEPs 
[Management  Decision  Packages].  We  used  the  percentage  of  total  O&M  allocated 
to  the  operational  unit  PEs  to  estimate  Army  operational  unit  collective  training 
costs.  We  used  the  ratio  for  Army  operational  units  for  our  estimation  of  Marine 
Corps  ground  units. 

We  compared  OPTEMPO  (Flying  hours)  funding  from  Navy  OP-20  and 
DoNSched  reports  to  total  O&M  to  establish  a  ratio  for  estimating  unit  collective 
training  for  Navy,  Marine  and  Air  Force  aviation  operational  units. 

Ships  unit  collective  training  costs  were  estimated  by  comparing  OP-41  steaming 
hours  costs  to  total  O&M  in  ships  PEs. 

We  estimated  that  30%  of  "Much  Support"  O&M  and  10%  of  "Some  Support" 
O&M  are  costs  of  unit  collective  training.  We  believe  that  this  provides  a 
reasonable  starting  point  for  evaluation  unit  training  costs  from  the  FYDP.  PEs  do 
not,  however,  provide  sufficient  detail  to  distinguish  unit  training  costs.  It  is 
conceivable  that  these  estimates  can  be  refined  by  the  services." 

Logistics  Management  Institute,  1993,  p  .1^ 

The  results,  shown  in  Figure  II-8,  are  the  only  known  data  that  attempt  to  estimate 
the  amount  of  funds  that  support  unit  training.  According  to  LMI's  estimate,  about 
$12.7  billion  was  allocated  to  unit  training  in  FY  1993.  The  largest  share  ($5.3  billion  or 
42  percent)  of  the  funds  identified  with  unit  training  are  allocated  to  the  Air  Force.  This 


^  The  data  developed  by  the  Logistics  Management  Institute  on  unit  training  by  Service  and  on  total 
OPTEMPO  by  Service  were  provided  by  the  Director,  Readiness  and  Training,  OUSD  (Personnel  and 
Readiness).  We  appreciate  the  assistance  provided  by  Michael  A.  Parmentier  and  Daniel  E.  Gardner  of 
OUSD  (P&R)  and  Steven  Liebennan  of  LMI. 


indicates,  again,  that  the  largest  expense  for  training  (unit  training  in  this  instance)  is  due  to 
aviation;  a  figure  that  includes  OPTEMPO  for  flying  hours  in  all  Services  is  given  below. 


Figure  11-8.  Estimated  Cost  of  Unit  Training  by  Service,  Speciai  Operations 
Command  and  Joint  Forces,  FY  1993 
(Source:  Logistics  Management  Institute,  1993) 

A  major  caveat:  these  numbers  represent  programmed  funds.  In  O&M,  more  than 
in  any  other  major  budget  category,  local  commanders  have  the  authority  to  transfer  funds 
from  one  Program  Element  to  another.  Hence,  it  is  possible  that  the  support  for  unit 
training  can  be  greater  than  or  less  than  that  indicated  in  this  figure,  even  if  the  estimated 
shares  of  programmed  funds  in  support  of  unit  trairung  should  happen  to  be  exactly 
correct.  For  example,  a  commander  who  puts  a  high  priority  on  a  scheduled  collective 
training  event  for  which  there  are  no  remaining  funds  has  the  authority  to  divert  money 
(e.g.,  from  maintenance  of  equipment  or  teal  property  to  unit  training)  and  can  in  this  way 
exceed  programmed  expenditures.  Another  commander  may  find  it  impossible  to  put  off 
certain  maintenance  for  which  he  has  insufficient  funding,  and  can  cancel  certain  training 
events  to  find  a  source  for  the  needed  funding.  Thus,  the  true  amount  of  support  for  unit 
training  can  only  be  known  ex  post  facto.  Neither  the  Comptroller  nor  any  other  agency  in 
OSD  has  the  means  to  track  O&M  expenditures  on  an  ongoing  basis.  It  would  requite  a 
very  time-consuming  data  call  to  the  Services  to  find  out  precisely  how  much  money  was 
actually  spent  on  unit  training  in  any  budget  year.  For  the  present,  the  informed  judgments 
produced  by  LMI  provide  the  best  available  assessment  of  unit  training  costs. 
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The  amount  of  field  training  that  each  Service  conducts  each  year  is  called 
OPTEMPO  (Operating  Tempo),  expressed  in  terms  of  the  average  number  of  flying  hours 
per  aircraft,  steaming  days  per  ship,  or  miles  traveled  per  vehicle.  The  costs  of  OPTEMPO 
include  the  costs  of  fuel,  consumption  of  spare  parts  and  maintenance-related  activities 
associated  with  training.  The  costs  of  these  activities  for  FY  1993-1995  were  estimated  by 
LMI  from  information  submitted  by  the  Services  in  the  Program  Objective  Memorandum 
(POM)  for  FY  95-99.  The  total  cost  of  OPTEMPO,  by  Service,  for  FY  1994  is  about 
$9.4  billion,  an  increase  of  $2.9  billion  (45  percent)  over  FY  1993  (Figure  II-9).  The 
Army  is  the  largest  Service  in  terns  of  personnel,  but  it  accounts  for  less  than  20  percent  of 
total  OPTEMPO  dollars.  The  Air  Force,  with  its  costs  for  the  use  of  aircraft,  accounts  for 
39  percent  of  the  total  OPTEMPO  budget. 


FY93 

FY94 

FY95 

Army 

$1.9  B 

$1.8  B 

$1.8  B 

Navy 

1.5 

3.1 

2.9 

Marine  Corps 

.1 

.8 

.8 

Air  Force 

3.0 

3.7 

3.7 

Total 

6.5 

9.4 

9.3 

Figure  11-9.  Cost  of  OPTEMPO,  by  Service,  FY  93-95 
(Source:  Logistics  Management  institute,  1993) 


Another  and  larger  estimate  of  OPTEMPO  is  provided  by  Angier,  Alluisi,  and 
Horowitz  (1992)  who  used  aggregated  budget  data  for  Service  expenditures  for  operations, 
consumables,  repairable  spares,  and  maintenance.  They  estimate  that  OPTEMPO  costs  for 
FY  1991  were  $21.4  billion,  as  follows: 


Flying  hours  (A,  N,  AF) 

$11.7  billion 

Steaming  days  (N) 

8.4 

Vehicle  miles  (A) 

_ L2 

Total 

21.4 

This  amount  is  2.3  times  larger  than  the  LMI  estimate  of  $9.4  hill  inn  for  FY  94  and 
some  explanation  of  the  difference  is  needed.  Since  both  studies  used  the  same  data 
sources  (i.e.,  the  Service  POMs)  the  difference  must  be  attributed  to  costs  included  in  the 
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larger  estimate  (Angier  et  al.)  that  were  not  included  in  the  smaller  estimate  (LMI).  For 
example,  Angier  et  al.  (1992)  included  costs  of  repair  and  depot  maintenance  that  are  not 
included  in  the  LMI  estimate.  We  were  not  able  to  account  for  other  items  that  could 
explain  the  different  results  of  the  two  studies,  even  after  contacting  the  authors. 
Obviously,  the  ad  hoc  nature  of  the  two  studies  shows  that  there  is  a  problem  of  defining 
the  cost  elements  that  should  (or  should  not)  be  included  in  the  costs  of  OPTEMPO  and  of 
identifying  them  in  each  report.  We  note,  with  pleasure,  that  this  problem  might  not  arise  if 
only  one  report  had  been  found.  The  absence  of  a  periodic  reporting  requirement  leads  to 
the  problem  we  observed  both  for  knowing  what  the  actual  costs  of  unit  training  and 
OPTEMPO  are  and  for  being  able  to  discern  trends  in  these  expenditures  that  may  have 
significant  impact  on  readiness. 

Neither  of  these  estimates  includes  certain  other  costs  that  are  clearly  relevant  to 
OPTEMPO,  such  as  transportation  of  equipment  and  personnel  from  home  base  to  the 
exercise  area  and  return  (to  the  extent  that  this  is  not  already  included  in  the  cost  of  fuel), 
ammunition  expended  during  training,  and  the  pay  and  allowances  of  personnel  being 
trained.  These  are  not  trivial  items  and  a  case  could  be  made  to  include  them,  or  not,  for 
various  applications.  But  the  critical  point  is  that  one  has  to  be  explicit  about  the  costs  that 
are  included,  with  a  definition  of  each  cost  element,  together  with  an  explanation  of  its 
relevance.  Unless  reliable  and  consistent  data  on  the  costs  of  OPTEMPO  and  unit  training 
are  reported  regularly,  it  is  difficult  to  believe  that  there  can  be  an  accurate  understanding 
about  the  costs  of  training  and  simulation  and  of  the  influence  of  these  various  cost 
elements,  if  any,  on  significant  outputs  such  as  Service  and  joint  readiness. 

The  Chairman  of  the  Joint  Chiefs  of  Staff  is  responsible  for  the  joint  Exercise 
Program  that  is  estimated  to  cost  about  $425  million  per  year.  About  75  percent  of  this 
amount  is  allocated  to  transportation  of  equipment  and  personnel  for  participation  in 
exercises.  A  worthy  purpose  can  be  served  to  the  extent  that  distributed  interactive 
simulation  provides  a  means  to  accomplish  joint  exercises  while  avoiding  the  costs  of 
transportation,  which  constitute  such  a  large  fraction  of  the  total  costs  of  exercises. 

Research  and  Development 

Research  and  development  on  Training  and  Personnel  Systems  Technology  (TPST) 
is  reported  in  four  functional  categories  (called  "Congressional"  categories): 

Manpower  and  personnel 

Education  and  training 
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Simulation  and  training  devices 
Human  factors  and  safety 

Data  on  programmed  expenditures  in  these  four  areas,  by  Program  Element  and 
Project  Number,  are  compiled  each  year  by  the  Manpower  and  Training  Research 
Information  System  (MATRIS),  San  Diego,  California,  an  agency  of  the  Defense  Technical 
Information  Center,  Alexandria,  Virginia.  The  total  expenditure  for  TPST  was  $287 
mUlion  in  FY  1993  and  $278  million  in  FY  1994  (Figure  IMO).  The  largest  expenditure 
among  the  four  categories  for  these  years  was  for  Simulators  and  Training  Devices:  $122 
milUon  (42  percent)  in  FY  1993  and  $125  miUion  (45  percent)  in  FY  1994.  Over  the  entire 
period,  the  Navy  spent  about  half  as  much  for  R&D  on  simulation  and  training  devices  as 


the  other  Services. 


Figure  11-10.  R&D  for  Simulation  and  Training  Devices,  by  Service, 
FY  1985-1994  (Source:  MATRIS,  1993) 


Total  R&D  for  simulation  and  training  devices  declined  to  a  low  of  about 
$88  million  in  FY  1990.  This  was  due  primarily  to  a  reduction  in  the  acquisition  of 
simulators  by  the  Air  Force.  The  Army  and  the  Air  Force  now  spend  close  to  the  same 
amount  ($65.6  and  $58.5  M,  respectively)  while  the  Navy  spends  less  ($18.7  M). 
(MATRIS,  1993) 

We  reviewed  all  projects  reported  by  MATRIS  in  the  two  categories  that  deal  with 
training:  (1)  Education  and  Training  and  (2)  Simulation  and  Training.  Work  unit  descrip¬ 
tions  were  examined  to  identify  those  concerned  primarily  with  technologies  relating  to 
training  equipment  or  methods  of  training  and  the  agencies  and  funds  assigned  to  each. 
The  results  are  summarized  in  Figure  II- 1 1,  while  the  details  are  shown  in  Appendix  B. 
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Training  Equipment 

Training  Methods 

Total  (M) 

Army 

STRICOM1 

$51.5  M 

AR|4 

$14.1  M 

$65.6 

Navy 

NTSC2 

16.6 

NADCS 

2.1 

18.7 

Air  Force 

TSSP03 

32.6 

AL/HRAT6 

22.2 

54.8 

TOTAL 

100.7 

38.4 

139.1 

^  STRICOM:  Army  Simulation  and  Training  and  Instrumentation  Command,  Orlando,  FL 
2  NTSC:  Naval  Training  Systems  Center,  Orlando,  FL,  now  Training  Systems  Division,  Naval  Air  Warfare 


Center,  Orlando,  FL 

3  TSSPO:  Air  Force  Training  Systems  Special  Program  Office,  Aeronautical  Systems  Division,  Wright- 
Patterson  Air  Force  Base,  OH 
*  ARI:  Army  Research  Institute,  Alexandria,  VA 
5  NADC:  Naval  Air  Development  Center,  Warminster,  PA 

®  AL/HRAT:  Air  Force  Armstrong  Laboratory,  Human  Resources  Directorate,  Air  Training  Research 
Directorate,  Mesa,  AZ 


Figure  11-11.  Funds  Allocated  to  Development  of  Training  Equipment  and 
Methods  of  Training,  by  Agency,  FY  1994  (Source:  MATRIS  1993) 

It  is  a  matter  of  interest  to  observe  that  $101  million  (72  percent)  of  the  funds 
allocated  to  simulation  and  training  in  FY  1994  are  assigned  to  the  training  equipment 
agencies  of  the  Services  (STRICOM,  NTSC,  and  TSSPO)  and  that  $38  million  (28 
percent)  are  assigned  to  the  training  research  and  development  agencies  (ARI,  AL/HRAT, 
and  NADC),  i.e.,  agencies  concerned  with  how  best  to  use  training  devices. 


On-the-Job  Training 

Although  it  is  obvious  that  in  order  to  achieve  full  proficiency  some  amount  of  on- 
the-job  training  is  needed  after  completion  of  formal  training  at  school,  the  effectiveness 
and  costs  of  on-the-job  training  have  received  only  limited  attention.  The  Navy  provides 
specialized  training  for  certain  jobs  in  two  ways:  at  formal  schools  for  periods  of  six  to  37 
weeks  (depending  on  rating),  or  by  on-the-job  training  in  the  fleet,  without  prior  schooling. 
Both  groups  receive  some  on-the-job  training  before  becoming  eligible  to  take  a  written 
examination  required  for  promotion  to  E-4  (third  class).  Weiher  and  Horowitz  (1971) 
compared  the  effectiveness  and  cost  of  each  type  of  training  for  the  same  job.  The  costs  of 
on-the-job  training  include  the  costs  of  supervision,  lower  productivity  of  trainees,  and  pay 
and  allowances  of  students  while  they  are  being  trained;  the  costs  of  training  at  schools 
includes  salaries  of  students  and  trainers  as  well  as  other  expenses  of  running  a  school. 
Weiher  and  Horowitz  found  that  those  who  have  only  on-the-job  training  take  an  average 
of  11.7  months  (72  percent  longer)  to  reach  full  proficiency  (qualification  to  take  the 
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promotion  examination)  compared  to  6.8  months  for  those  who  attended  Apprentice  ("A") 
school;  the  comparable  costs  are  $384  million  (66  percent  higher)  for  OJT  and 
$23 1  million,  for  those  who  attended  school  (data  for  a  six-month  test  cycle).  These 
findings  do  not  mean  that  school  training  can  replace  on-the-job  training.  However, 
because  of  the  higher  cost  of  on-the-job  training,  determination  of  an  optimum  mix  of  both 
methods  of  training,  for  particular  jobs,  would  seem  to  be  a  worthy  goal  that  has  received 
little  attention. 

Not  only  is  the  cost  of  on-the-job  training  almost  invisible,  but  it  may  even  be  said 
not  to  cost  anything  as  long  as  manpower,  when  not  at  war,  is  available  to  provide  OJT. 
Of  course,  the  argument  against  this  view  is  that  there  is  an  implied  cost  for  personnel 
needed  to  conduct,  or  supervise,  on-the-job  training  of  new  personnel,  and  who  are  not, 
therefore,  able  to  perform  their  other  duties.  Low  productivity  of  trainees,  pay  and 
allowances  of  trainees  and  supervisors,  are  other  elements  of  the  costs  of  OJT. 

Gay  (1974)  developed  a  method  of  estimating  the  costs  of  on-the-job  training  in  the 
Air  Force,  based  on  an  application  of  modem  human  capital  theory;  investment  in  OJT  is 
measured  as  the  present  value  of  the  sum  of  positive  differences  between  an  individual's 
military  pay  and  productivity  (in  dollars)  over  time.  In  a  pilot  study  of  Aircraft  Main¬ 
tenance  Specialists,  OJT  was  estimated  to  cost  $6,600  per  year  (FY  1972)  for  trainees  who 
also  had  attended  Air  Force  technical  school,  which  cost  an  additional  $3,200.  In  this  case, 
training  of  Aircraft  Maintenance  Specialists  cost  a  total  of  about  $9,800  over  four  years,  of 
which  OJT  cost  about  two-thirds  or  twice  as  much  as  prior  school  training. 

Shiells  (1991)  studied  13  non-technical  Naval  ratings  that  can  be  trained  either  at 
school  or  on-the-job.  She  found  about  the  same  level  of  performance  for  personnel  trained 
both  ways,  with  a  tendency  that  graduates  of  longer  courses  (with  a  maximum  of  three 
months)  performed  better  than  graduates  of  shorter  courses.  In  this  study,  effectiveness  of 
on-the-job  training  was  assessed  by  comparing  the  survival,  promotion,  and  re-enlistment 
rates  of  personnel  qualified  on  the  job  to  those  who  qualify  by  attending  apprentice  ("A") 
school. 

Even  when  on-the-job  training  is  required  to  improve  the  skills  learned  at  school, 
the  operational  environment  is  often  not  a  congenial  one  for  training.  Operational 
assignments  have  priority  over  those  related  to  training  and  the  conditions  needed  for 
adequate  training  may  not  be  available.  Semb,  Ellis  et  al.  (1993)  found  that  very  few 
shipboard  petty  officers  receive  any  training  or  information  on  how  to  conduct  on-the-job 
training.  While  over  50  percent  of  supervisors  reported  that  they  used  appropriate  OJT 


techniques,  at  least  20  percent  said  that  they  do  not  and  another  30  percent  are  not  as 
effective  as  they  could  be.  Since  these  data  were  collected  by  means  of  a  questionnaire, 
these  findings  may  be  viewed  as  optimistic  and  actual  performance  of  OJT  is  probably 
poorer  than  is  reported  here.  Figure  11-12,  taken  from  Gay  and  Nelson  (1974),  is  a 
notional  representation  of  the  fact  that  both  training  at  school  and  on-the-job  have  real  costs 
and  that  the  returns  that  new  personnel  make  to  the  military  service  become  positive  only 
after  some  period  of  learning.  The  cost  of  training  and  length  of  the  period  of  learning  is  a 
function  of  the  complexity  of  various  jobs;  formal  training  lasts  more  than  18  months  for 
some  electronics  and  nuclear  technicians  occupations.  The  costs  of  on-the-job  training 
include  the  pay  and  allowances  of  students  and  of  those  who  supervise  their  work  as  well 
as  damage  that  novices  may  cause  to  tools  and  equipment,  less  the  positive  contribution  that 
their  increasing  productivity  brings  to  the  job. 


Quester  and  Marcus  (1984)  compared  the  cost  of  training  Navy  technicians  either  at 
school  or  on-the-job  in  12  occupations  and  their  productivity  over  four  years.  Net 
productivity  of  trainees  was  rated  by  supervisors  for  four  points  in  time  (1  month,  and  1, 
2, 4  years)  as  a  percent  of  the  output  of  an  average  specialist  with  four  years  of  experience 
in  that  occupation;  net  productivity  is  an  estimate  of  the  contribution  to  work  minus  the  loss 
in  production  of  experienced  personnel  who  train  and  supervise  trainees.  In  all  occupations 
(e.g..  Aviation  Machinists'  Mate,  Radioman,  Electronics  Technician),  personnel  trained  at 
school  were  more  productive  (by  an  average  of  27  percent)  than  those  trained  on  the  job 
(Figure  11-13).  For  most  occupations,  graduates  of  formal  schooling  are  more  cost- 
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effective  than  those  trained  exclusively  on  the  job;  the  reverse  is  true  for  a  few  less  technical 
jobs  such  as  Electronics  Technician,  Radiomen,  and  Machinist's  Mates.  School  training 
would  appear  to  be  even  more  cost-effective  if  data  were  collected  to  cover  longer  careers 
than  the  first  enlistment  (i.e.,  more  pay-back). 


Figure  11-13.  Productivity  Growth  for  Eiectricians'  Mates  in  the  First  Term 
(Source:  Quester  and  Marcus,  1984) 

New  technology,  such  as  distance  learning,  interactive  courseware,  multi-media 
training  devices,  computer-based  instruction,  as  well  as  portable,  sophisticated  electronic 
job  aids  for  maintenance  (e.g.,  Vu-Man)  can  significantly  improve  the  effectiveness  of  the 
informal  type  of  training  that  occurs  in  operational  units;  interactive  videodisc  instruction 
reduces  training  time  by  30  percent  (Fletcher,  1990);  Vu-Man  cuts  the  time  of  technical 
inspection  of  equipment  by  50  percent  (Deutch,  1994).  The  collection  of  reliable  data  on 
the  cost  and  effectiveness  of  on-the-job  training  with  current  and  new  tools  is  a  difficult 
task  but  it  is  still  needed  to  assess  the  utility  of  this  type  of  training. 

For  reasons  that  are  not  obvious,  OJT  remains  a  generally  overlooked  type  of 
training  whose  effectiveness  could  be  improved  and  whose  current  costs  are  not  knotvn. 

Discussion 

Data  on  the  costs  of  simulation  and  training  that  have  been  presented  in  this  chapter 
are  useful  but  incomplete.  Data  on  the  cost  of  individual  training  in  institutions  are 
considered  to  be  reliable  and  valid.  They  are  reported  regularly  in  the  annual  Military 
Manpower  Training  Report,  so  that  trends  in  the  number  of  people  trained  in  various  types 
of  training  and  their  costs  are  known;  the  cost  categories  used  in  this  report  have  been 


defined,  ate  used  consistently,  and  have  an  average  error  of  less  than  one  percent  (Elder  et 
al.,  1986, 1989).  A  similar  situation  exists  in  two  other  areas:  the  content  and  cost  of  the 
research  and  development  program  on  Training  and  Personnel  Systems  Technology 
(reported  by  the  Manpower  and  Training  Research  Information  System,  or  MATRIS,  an 
agency  of  the  Defense  Technical  Information  System);  and  on  the  procurement  of  military 
trainers  and  simulators  (a  commercial  market  survey  produced  from  time  to  time  by  Frost  &. 
Sullivan  Market  Intelligence,  New  York,  N.Y.).  Information  contained  in  the  latter  survey 
is  compiled  from  information  published  by  the  Department  of  Defense  on  prime  contract 
awards  and  on  summaries  of  planned  procurements  provided  by  the  Services.  Frost  & 
Sullivan  provided  a  convenient  and  quick  way  to  collect  information  that  would  otherwise 
require  an  effort  beyond  our  means  for  this  study.  It  is  not,  obviously,  an  authoritative 
document  issued  by  the  Department  of  Defense. 

Little  documentation  of  this  quality  applies  to  our  other  data  on  the  costs  of  training 
and  simulation.  All  of  our  sources  are  the  result  of  ad  hoc  efforts,  and  these  reports  do  not 
always  identify  or  define  the  cost  elements  in  a  consistent  (or  even  clear)  manner;  some  of 
our  data  were  provided  by  briefing  material  or  informally  by  professional  colleagues.  This 
makes  it  difficult,  and  perhaps  unwise,  to  compare  or  combine  data  from  different  sources 
in  order  to  analyze  the  cost  and  effectiveness  of  various  types  of  training  and  simulation 
even  though  such  analyses  are  vital  for  developing  policy  to  guide  research  on  training  and 
simulation  towards  areas  of  maximum  pay-off.  The  reader  is  therefore  warned  to  use  our 
cost  data  with  care  even  though,  we  believe,  they  are  the  most  extensive  currently  available 
on  simulation  and  training.  It  is  also  difficult  to  comprehend  how  policy  on  the  technology 
most  relevant  to  simulation  and  training,  and  to  the  procurement  of  equipment  and 
procedures  for  the  most  effective  use  of  simulation  and  training  can  be  developed 
meaningfully  in  the  absence  of  more  complete  and  reliable,  consistent  and  comparable  data 
on  the  costs  of  these  activities. 

Mindful  of  the  qualifications  that  must  apply  to  the  data  we  found,  we  will  examine 
these  costs  in  two  ways: 

(1)  Annual  expenditures 

(2)  Type  and  location  of  training. 

Annual  Expenditures 

The  cost  data  shown  in  previous  figures  cover  different  periods  of  time.  In 
Figure  11-14,  these  costs  are  reported  as  annual  expenditures  for  RDT«&E,  initial 
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Type  of  Expenditure 

Amount  1 

RDT&E 

Simulators  for  weapon  systems 

$0,336 

Technology 

training  equipment 

0.101 

training  methods 

0.038 

Modeling  and  simulation 

Joint  commands 

0.019 

DMSO 

0.073 

ARPA 

0.103 

Initial  investment 

Procurement  of  simulators 

Aviation  0.800 

Non-system  devices  0.243 

All  others  0.057 

1.100 

Models  and  simulations 

NA 

Military  Construction 

NA 

individual  training  in  institutions 

14.4 

Individual  training  in  units 

NA 

Collective  training  in  institutions 

NA 

Collective  training  in  units 

12.7 

OPTEMPO 

9.4 

21.4 

Joint  exercises 

0.425 

Simulator  maintenance 

0.369 

Period 


Source 


FY91-97  Frost  &  Sullivan  (1993)2 


MATRIS  (1993)3 
MATRIS  (1993)3 

IG  (1993)4 
DMSO  (1994)5 
ARPA  (1993)6 


FY  91-97  Frost  &  Sullivan  (1993) 


FY91 
FY94 
FY  91-97 


MMTR  (1993)7 


LMI  (1993)8 

LMI  (1993) 
Angier  et  al.  (1992)® 

Briefing  material 
Frost  &  Sullivan  (1993) 


’  Amount,  in  billions,  for  year  cited  or  average  of  years  noted. 

^  Frost  &  Sullivan  (1 993) 

3  Manpower  and  Training  Research  Information  System  (1993) 

^  Inspector  General  (1993) 

^  Defense  Modeling  and  Simulation  Office  (1 994) 

®  Advanced  Research  Projects  Agency  (1993) 

^  Milrtary  Manpower  Training  Report,  FY  94  (1993) 

®  Logistics  Management  InstKute  (1993) 

®  Angier,  Alluisi,  and  Horowitz  (1 992) 

NA:  Not  Available 

Figure  il-14.  Estimated  Annual  Expenditures  for  Training  and  Simuiation,  in 
Terms  of  RDT&E,  initial  Investment,  and  Operating  and  Support  of  Training 
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investment,  and  operating  and  support,  either  for  the  year  noted  or  as  an  average  of  the 
years  for  which  costs  are  available^  the  format  follows  the  cost  element  structure  for 
defense  training  developed  by  Knapp  and  Orlansky  (1983)  and  other  studies  of  defense 
costs.  One  is  advised  not  to  add  these  costs  since  there  are  the  possibilities  both  of 
redundancy  and  incompleteness.  Nevertheless,  it  is  clear  that  the  annual  costs  of  training 
(operating  and  support)  are  in  the  tens  of  billions,  procurement  of  training  equipment 
(initial  investment)  in  the  billions,  and  research  and  development  on  training  equipment  and 
training  methods,  and  on  modeling  and  simulation  (including  distributed  interactive 
simulation)  also  in  the  order  of  millions;  the  latter  costs,  while  small  in  comparison,  may  be 
incomplete.  We  were  not  able  to  find  cost  data  on  individual  training  in  units  (which  could 
be  large),  or  on  investment  costs  for  modeling  and  simulation  and  military  construction  for 
training,  or  on  collective  training  in  units.  AVhether  these  costs  include  costs  for 
transportation  to  training  sights,  ammunition,  and  other  incidental  costs  related  to  training  is 
unknown. 

Type  and  Location  of  Training 

Figure  11-15  follows  the  paradigm  suggested  by  General  Paul  Gorman  in  Chapter  I 
and  reports  the  costs  of  training  shown  in  the  preceding  figure  on  the  basis  of  type  of 
training  and  where  training  occurs.  Individual  training  at  institutions  cost  $14.4  billion  in 
FY  1994;  collective  training  in  units  cost  $12.7  billion  in  FY  1994,  while  OPTEMPO,  also 
in  units,  is  estimated  to  cost  $9.4  billion  (FY  1993)  or  $21.4  billion  (FY  1991),  according 
to  two  different  sources  for  this  data.  We  have  determined  that  the  larger  figure  includes 
costs  of  depot  maintenance  that  are  not  included  in  the  smaller  figure;  but  this  cannot 
account  for  all  of  the  large  difference.  It  is  also  important  to  note  that  the  costs  of  collective 
training  in  institutions  and  of  individual  training  in  units  are  not  now  available.  The  raw 
data  undoubtedly  exist  but  there  are  no  present  means  to  define  the  relevant  cost  elements 
and  locate  their  sources,  other  than  to  commission  another  ad  hoc  study.  Nor  is  there  a 
report  that  compiles  such  costs  periodically  to  develop  trends  in  expenditures  and  provide  a 
basis  for  oversight  and  policy  development.  As  was  pointed  out  previously,  the  cost  data 
on  unit  training  and  OPTEMPO  are  the  result  of  ad  hoc  efforts  that  require  clarification  to 
be  truly  useful.  Data  on  RDT&E  and  on  initial  investments  for  training  are  not  included  in 
this  figure. 


Where  Training  Occurs 


Institution 

Unit 

Who  is 

trained 

Bv  service.  FY  94^ 

Army 

Navy 

Marine  Corps 

Air  Force 

Bvtvpe,  FY  94'' 

$5.4  B 
4.5 

1.3 

_32. 

14.4 

Not  available; 

Individual  (on-the-job)  training  in 
units  for  those  trained  only  on  the 
job,  job  familiarization  for  those 

individual 

Recruit  $1 .2  B 

Officer  acquisition  .5 

Specialized  skill  4.1 

Flight  22 

Professional  development  .9 
OSUT  (Army)7  .3 

14.4 

trained  at  schools,  and  training  for 
those  assigned  jobs  that  do  not 
match  their  MOS® 

Unit  training  $12.7  FY  93^ 

Collective 

Army  Combat  Training 
Centers,  FY  95 

.5  B® 

OPTEMPO  9.4  FY  93^ 

21.4  FY913 

Other  data  not  available 

Joint  exercises  0.425  FY  94'* 

1MMTR  (1993). 

2LMI  (1993). 

3  Angler,  Alluisi,  and  Horowitz  (1992). 

*  Briefing  Estimate. 

5  AUSA  (1994);  about  one-third  of  this  total  is  for  transport  to  and  from  the  training  center  (Fig.  11-16). 
®  CBO  (1994)  estimates  6-10  percent  mismatch  between  job  and  MOS  in  the  Army.  FY  1993  (p.  32). 
7  OSUT:  Army  One-Station  Unit  Training:  combines  Recrurt  and  Specialized  skill  training. 


Figure  IMS.  Estimated  Annual  Expenditures  for  Training,  by  Type  of  Training 
and  Where  Training  Occurs  (exciudes  RDT&E  and  initiai  investment) 
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The  significance  of  including  other  costs,  such  as  for  transport,  pay,  and  ammuni¬ 
tion,  may  be  illustrated  by  the  Angier  et  al.  (1992)  estimate  of  the  costs  of  one  field  training 
exercise  by  an  armored  battalion  at  the  National  Training  Center  (Figure  11-16);  the  total 
cost  is  estimated  to  be  $16.25  million.  A  visit  by  an  armored  battalion  to  the  NTC  takes 
about  one  month.  Half  of  that  time  is  spent  in  the  actual  exercise,  while  the  rest  of  the  time 
is  spent  packing  and  unpacking  equipment  and  traveling.  Transport  accounts  for 
31  percent  and  pay  and  allowances  of  the  OPFOR  for  19  percent  of  the  total  cost  of  one 
rotation;  these  data  include  but  do  not  specify  separately  the  cost  of  ammunition;  pay  and 
allowances  of  the  BLUFOR,  the  battalion  being  trained,  are  not  included.  It  is  helpful  to 
know  precisely  what  costs  are  or  are  not  included.  In  saying  so,  we  do  not  suggest  that 
there  is  one  right  or  best  way  to  present  cost  data  on  training  because  the  particular  analysis 
should  dictate  the  costs  that  are  considered  to  be  relevant  to  the  analysis.  Our  inability  to 
estimate  the  overall  costs  of  unit  training  and  OPTEMPO  is  a  result  of  uncertainty  about  the 
composition  of  the  data  we  were  able  to  find. 


Transport  (base  to  NTC  and  back) 

$  5.0  million 

Tools  and  spares 

.5 

Range  decontamination 

.25 

OPFOR  OPTEMPO  and  ammunition 

4.5 

Blue  OPTEMPO  and  ammunition 

3.0 

OPFOR  pay  and  allowances 

3.0 

TOTAL 

16.25 

Figure  11-16.  Cost  of  One  Exercise  at  the  Nationai  Training  Center 
(Source:  Angier  et  ai.,  1992,  p.  A-7) 


FINDINGS 

1 .  Data  on  the  costs  of  simulation  and  training  and  of  its  major  components  are 
not  now  reported  in  any  regular  or  consistent  fashion.  Two  major  exceptions  are  the  costs 
of  individual  training  at  institutions,  which  are  reported  in  the  Military  Manpower  Training 
Report  and  the  costs  of  Training  and  Personnel  Systems  Technology,  which  are  reported 
by  the  Manpower  and  Training  Research  Information  System.  Estimates  of  the  costs  of 
collective  training  in  units,  OPTEMPO,  joint  exercises,  procurement  of  simulators,  R&D 
on  modeling  and  simulators  were  found  in  some  ad  hoc  studies.  Apart  from  the  fact  that 
these  costs  are  not  reported  regularly,  nor  are  readily  available,  such  reports  are  often  not 
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clear  about  the  cost  elements  that  are  included  or  are  explicitly  excluded.  Important  data  on 
training  costs  that  could  not  be  found  include  the  costs  of  individual  training  in  units  (on- 
the-job  training),  collective  training  in  institutions,  Service  exercises,  the  procurement  of 
simulators  (a  commercial  source  was  used  for  these  data),  and  modeling  and  simulation. 
All  of  the  raw  data  on  such  costs  undoubtedly  exist  somewhere  within  the  Services  and  in 
various  data  banks;  however,  they  are  not  reported  regularly  in  a  form  useful  for  evaluating 
the  cost  and  effectiveness  of  various  methods  of  simulation  and  training,  and  for  policy 
decisions  on  developing  the  required  technology  and  on  investing  in  the  most  cost-effective 
simulation  and  training  systems. 

2.  The  annual  cost  of  individual  training  at  institutions  is  estimated  to  be 
$14.4  billion  (FY  1994).  This  value  comes  from  the  Military  Manpower  Training  Report, 
an  annual  report  that  is  clearly  reliable.  The  annual  cost  of  collective  training  in  units  is 
estimated  in  one  report  to  be  $12.7  billion  (FY  1993).  OPTEMPO  (operating  tempo  in 
units)  is  estimated  in  one  study  to  cost  $9.4  (FY  1993)  and  $21.4  billion  (FY  1991)  in 
another.  These  values  cannot  include  the  same  cost  elements;  but,  except  for  one  element 
(depot  maintenance),  we  were  not  able  to  explain  the  reasons  for  such  widely  different 
estimates. 

3.  The  cost  of  procuring  simulators  is  estimated  to  be  about  $1.2  billion  per  year 
(FY  91-97);  RDT&E  for  simulators,  simulations,  modeling  and  methods  of  training  is 
estimated  to  be  about  $.6  billion  per  year  (FY  91-97).  These  estimates  come  from  different 
sources  and  need  further  examination. 

4.  Aviation  accounts  for  the  largest  expenditures  for  training  when  estimated  by 
the  costs  for  procurement  of  simulators,  initial  pilot  training,  or  OPTEMPO  for  flying 
hours.  There  is  little  evidence  that  research  and  development  on  training  is  concentrated  in 
these  areas  of  high  pay-off.  A  similar  observation  applies  to  research  and  development  on 
distributed  interactive  simulation,  where  large  costs  for  future  procurements  are  likely  to 
occur. 

5 .  On-the-job  training  is  a  neglected  area  with  regard  both  to  its  effectiveness  and 
its  cost.  Although  all  personnel  in  all  military  occupational  specialties  receive  some  on-the- 
job  training,  there  is  reason  to  believe  that  its  costs  are  obscure  (because  of  the  efforts 
required  to  supervise  trainees)  and  that  there  is  room  to  improve  the  efficiency  of  this 
method  of  training  (by  the  use  of  new  technology,  such  as  distance  learning,  computer- 
based  instruction,  and  advanced  job-aiding  devices). 


11-24 


III.  COST  AND  EFFECTIVENESS  OF  SIMULATORS 


Training  in  a  simulator  is  known  to  improve  performance  in  that  simulator.  If  the 
training  transfers,  i.e.,  if  the  skills  learned  in  a  simulator  can  be  performed  in  the  actual 
equipment,  then  the  simulator  is  an  effective  training  device.  If  the  skills  do  not  transfer, 
then  the  improved  performance  in  the  simulator  merely  means  that  the  trainee  has  learned  to 
master  the  simulator  and  its  quirks.  What  we  really  want  to  know  is  the  extent  to  which 
training  in  a  simulator  improves  (or  diminishes)  performance  in  the  actual  equipment  for 
which  the  simulator  is  supposed  to  provide  training. 

Many  studies  show  that  simulators  can  provide  cost-effective  training.  Briefly,  this 
means  that  simulators  train  as  well  as  actual  equipment  but  cost  less  to  buy  and  use.  But 
this  does  not  apply  to  all  tasks,  all  types  of  training,  and  to  simulations.  For  example,  it 
has  been  determined  that  the  SIMNET  system  has  the  capability  to  train  tank  crew  members 
on  about  35  percent  of  the  tasks  specified  in  the  Army  Training  Evaluation  Program 
(ARTEP)  (Burnside,  1990).  This  is  a  consequence  of  the  design,  technology,  and  cost 
targets  set  for  the  development  of  SIMNET.  Therefore,  the  effectiveness  of  SIMNET 
should  be  evaluated  on  the  basis  of  how  well  personnel  trained  in  SIMNET  can  perform 
these  35  tasks  in  tanks.  An  expectation  that  trainees  should  be  able  to  perform  more  or  all 
of  the  ARTEP  tasks  could  easily  show  that  SIMNET  has  low  effectiveness,  but  that  would 
be  an  erroneous  conclusion.  An  enlarged  standard  could  be  set  for  the  Close  Combat 
Tactical  Trainer  (CCTT),  an  advanced  version  of  SIMNET  where  it  has  been  determined 
that  the  new  device  can  provide  training  on  about  60  percent,  but  not  100  percent,  of  all 
ARTEP  tasks  (Noble  and  Johnson,  1991).  Clearly,  the  remaining  40  percent  of  the 
ARTEP  tasks  could  be  trained  either  in  the  actual  tank  or  in  another  simulator  designed  for 
that  purpose.  It  is  often  said  that  using  the  actual  equipment  is  the  best  way  to  train  but  this 
is  clearly  a  short-sighted  view.  In  some  cases  this  is  simply  too  dangerous  for  a  novice;  in 
others,  the  equipment  is  so  complex  that  some  other  way  of  training  must  precede  exposure 
of  a  trainee  to  the  actual  equipment;  that  is  why  the  military  uses  trainer  aircraft  and 
simulators. 

Two  issues  dominate  the  choice  between  using  a  simulator  for  training  and  using 
actual  equipment;  the  effectiveness  and  the  cost  of  using  the  simulator  for  training  as 
compared  to  the  effectiveness  and  cost  of  using  the  actual  equipment  for  the  same  purpose. 
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Effectiveness  means  the  level  of  performance  achieved  by  use  of  the  simulator  to  obtain  the 
skills  specified  as  needed  to  operate  the  actual  equipment  or,  in  the  case  of  a  maintenance 
training  simulator,  to  maintain  complex  equipment.  Cost  should  mean  all  costs,  on  a  life 
cycle  basis;  some  studies  examine  only  the  procurement  costs  or  the  operating  costs. 
Figure  ni-l  shows  the  possible  outcomes  in  a  comparison  of  the  cost  and  effectiveness  of  a 
simulator  versus  that  of  actual  equipment  for  training  and,  as  a  consequence,  the  decision 
that  should  be  made  for  each  outcome. 


EFFECTIVENESS 

LESS 

SAME 

MORE 

LESS 

7 

+ 

+ 

COST 

SAME 

- 

7 

+ 

MORE 

- 

- 

7 

+  ADOPT 
-  REJECT 
?  UNCERTAIN 


Figure  IIM.  Decision  Diagram  for  Evaluating  the  Effectiveness  and  Cost  of  Two 

Methods  of  Training 

Studies  that  have  evaluated  the  cost-effectiveness  of  flight  simulators  and  main¬ 
tenance  simulators  are  summarized  below.  Other  types  of  simulators,  such  as  gunnery 
trainers,  have  also  been  evaluated  but  primarily  for  effectiveness  and  not  cost;  such  studies 
do  not  yield  a  coherent  picture  and  are  not  considered  here.  To  the  best  of  our  knowledge, 
the  cost-effectiveness  of  other  types  of  simulators  has  not  been  systematically  evaluated. 
Because  the  Close  Combat  Tactical  Trainer  (advanced  SIMNET)  is  a  large  procurement,  a 
cost  and  operational  effectiveness  analysis  will  be,  but  has  not  yet  been,  conducted.  Many 
evaluations  of  flight  simulators  and  maintenance  simulators  have  been  performed. 


Flight  Simulators 

Except  for  aircraft  designed  specifically  for  use  as  trainers,  most  aircraft  are 
designed  to  perform  some  operational  requirement,  such  as  air  superiority  or  transport. 
Their  use  for  training  is  an  ancillary  function.  Thus,  it  is  relevant  to  compare  the  operating 
costs  of  using  a  combat  aircraft  or  a  simulator  of  that  aircraft  for  training  purposes,  and  not 
the  RDT&E  or  procurement  costs  that  are  directed  towards  combat  rather  than  training;  the 
costs  of  RDT&E  and  procurement  would  be  relevant  if  we  were  evaluating  an  airplane 
designed  primarily  for  purposes  of  training.  A  comparison  of  the  operating  costs  of  42 
flight  simulators  and  aircraft  is  shown  in  Figure  111-2. 


Figure  III-2.  Variable  Operating  Costs  per  hour  for  42  Flight  Simulators  and 
Aircraft  (Source:  Orlansky,  Knapp,  and  String,  1983) 

Variable  operating  costs  include  the  costs  of  fuel,  lubricants  and  spare  parts  needed 
to  maintain  equipment,  as  a  function  of  its  use.  The  data  show  that  the  cost  of  operating  a 
flight  simulator  varies  between  5  and  20  percent  that  of  its  comparable  airplane.  The 
median  value  of  several  such  assessments  is  about  10  percent  (Orlansky  and  String,  1977; 
Orlansky,  Knapp,  and  String,  1983). 
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Figure  III-3.  Transfer  Effectiveness  Ratios  from  22  Studies  (Source:  Orlansky 

and  String,  1977) 


The  Transfer  Effectiveness  Ratio  (TER)  compares  the  amount  of  training  time 
needed  to  perform  a  specific  task  in  an  airplane  either  (1)  after  training  only  in  the  airplane 
or  (2)  after  prior  training  in  a  simulator.  The  TER  describes  the  amount  of  flight  time 
saved,  as  a  function  of  the  time  spent  in  a  simulator  on  the  same  task.  A  ratio  of  one  would 
mean  that  one  hour  spent  in  a  simulator  saves  one  hour  in  the  airplane.  The  median  of  34 
TERs  compiled  from  22  studies  is  0.48;  this  means  that  about  one-half  hour  is  saved  in  the 
air  for  every  hour  of  prior  training  on  the  same  task  in  a  flight  simulator  (Figure  III-3). 
Positive  transfer  from  simulator  to  aircraft  is  also  reported  by  Hays  et  al.  (1992)  in  a  paper 
that  summarizes  the  effectiveness  of  flight  simulator  training  found  in  15  more  recent 
studies. 

Since  the  use  of  a  simulator  for  training  (1)  results  in  flight  performance  that  is 
about  the  same  as  that  produced  by  the  use  of  an  airplane,  (2)  saves  flight  time,  and  (3) 
costs  only  about  ten  percent  of  the  cost  of  an  airplane,  it  follows  that  a  flight  simulator  is  a 
cost-effective  method  of  flight  training  by  a  factor  of  about  five.  Considering  that  the 
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technology  of  flight  simulators  (e.g.,  computers,  visual,  and  motion  systems)  has 
improved  beyond  that  contained  in  these  studies,  and  that  their  costs  have  also  been 
reduced,  it  is  reasonable  to  believe  that  the  cost-effectiveness  of  currently  available 
simulators  has  probably  increased. 

Only  a  few  studies  have  examined  the  amortization  of  flight  simulators,  i.e.,  how 
long  it  takes  for  the  savings  attributed  to  their  use  to  pay  for  the  cost  of  procuring  them. 
Two  studies  suggest  that  the  cost  of  procuring  military  flight  simulators  can  be  amortized  in 
about  two  years  (Figure  ni-4).  A  similar  study  shows  that  a  major  airline  amortized  the 
cost  of  five  flight  simulators  in  less  than  one  year.  The  main  reason  for  the  difference  may 
be  found  in  the  utilization  rate;  commercial  airlines  use  their  simulators  for  almost  24  hours 
a  day  compared  to  8  to  16  hours  a  day  for  the  Services. 
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Simulator 

Procurement 

Cost 

Savings 
per  Year 

'N 

Time  to  Amortize 

Costs 

Coast  Guard,  HH-S2A 
HH-3F 

$3.1  M 

$  1.5  M 

2.1  years 

Navy,  P-3C 

4.2  M 

2.5  M 

1.7  years 

Airline 

17.5  M 

25. 3  M 

8.3  months 

^  Procurement  cost/savings  per  year 


Figure  III-4.  Amortization  of  Flight  Simulators 
(Source:  Orlansky  and  String,  1977) 

Although  actual  equipment  is  widely  used  for  training  maintenance  technicians, 
simulated  maintenance  equipment  and  test  bench  equipment  have  certain  advantages:  a 
wider  variety  of  malfunctions  can  be  represented;  malfunctions  can  be  "installed"  more 
readily;  critical  parts  can  be  exposed  or  blown  up  for  purposes  of  training;  and,  not  least, 
performance  can  be  measured.  Some  maintenance  simulators  look  almost  exactly  like  the 
actual  equipment 

The  studies  summarized  in  Figure  111-5  show  that  students  trained  on  maintenance 
simulators  can  perform  maintenance  tasks  about  as  well  as  those  trained  on  actual 


equipment.  A  few  studies  provide  data  on  student  time  savings,  if  any;  when  reported, 
time  savings  due  to  the  use  of  simulators  are  of  the  order  of  25  to  50  percent.  One  could 
argue  that  maintenance  personnel  trained  on  simulators  should  perform  better  than  those 
trained  on  actual  equipment.  The  reason  is  that  simulators,  although  artificial,  can  provide 
cues  to  malfunctions,  and  hints  to  diagnosis  of  faults  that  are  not  available  in  actual 
equipment  and  these  hints  make  it  easier  to  diagnose  real  malfunctions  when  actually 
needed.  However,  the  current  data  do  not  support  this  proposition. 
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Figure  ill-5.  Studies  on  the  Effectiveness  of  Maintenance  Simulators 
(Source:  Orlansky  and  String,  1981) 


Computer-Based  Instruction 

Computer-based  instruction  is  used  for  training  on  many  jobs  in  the  military 
services  and  industry  and  for  education  at  all  levels  from  kindergarten  to  graduate  school. 
No  effort  will  be  made  here  to  evaluate  the  growing  capability  of  what  is  sometimes  called 
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"high-tech  learning."  There  is  a  very  large  literature  on  this  subject.  Here,  we  briefly 
summarize  a  few  major  substantive  findings  of  interest  to  military  training;! 

1 .  Computer-based  instruction  saves  about  30  percent  ot  the  time  students  need  to 
complete  a  course,  compared  to  conventional  instruction. 

2 .  Interactive  videodisc  instruction  raises  performance  of  50th  percentile  students 
to  about  the  70th  percentile  achievement  level. 

3 .  The  average  cost  of  interactive  videodisc  instruction  is  about  40  percent  that  of 
conventional  instruction. 

Since  current  computers  are  easily  transportable  and  their  costs  have  been  sharply 
reduced,  they  have  high  value  for  individual  training  in  institutions  as  well  as  in  units,  their 
use  for  collective  training  also  appears  attractive. 


Figure  ni-6  summarizes  the  findings  of  studies  that  have  reported  data  on  the  cost- 
effectiveness  of  flight  simulators,  maintenance  simulators,  and  computer-based  instruction. 


SAVINGS  OR  COST 

EFFECTIVENESS 

FACTOR 

FLIGHT 

SIMULATOR 

COMPUTER- 

BASED 

INSTRUCTION 

MAINTENANCE 

SIMULATORS 

ABOUT 

STUDENT  TIME 
SAVINGS 

50%  OF 
SIMULATOR 
TIME 

30% 

20-50% 

THE 

ACQUISITION 

COST 

30-65% 

? 

20-60% 

SAME 

OPERATING 

COST 

10% 

? 

50% 

' 

LIFE-CYCLE 

COST 

65% 

? 

40% 

AMORTIZATION 

2  YEARS 

? 

4  YEARS 

Figure  III-6.  Summary  of  Findings  on  the  Effectiveness  and  Cost  of  Flight 
Simulators,  Computer-Based  Instruction,  and  Maintenance  Simulators 


Fletcher  (1990),  Fletcher  and  Orlansky  (1989),  Spencer  (1991),  Orlan.sky  and  String  (1977),  Kulik  and 
Kulik  (1986),  Kulik  et  al.  (1986) 
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These  simulators  are  as  effective  for  training  as  the  use  of  actual  equipment  and  cost 
less  to  own  and  to  operate.  Students  take  less  time  to  master  lessons  when  they  are  trained 
with  simulators  than  with  actual  equipment;  the  average  time  savings  is  about  30  percent. 
The  acquisition  cost  of  simulators  can  be  amortized  in  two  to  four  years.  Most  of  these 
findings  are  based  on  studies  of  initial  rather  than  advanced  training. 

Discussion 

Flight  simulators,  maintenance  simulators,  and  computer-based  instruction  are 
demonstrably  cost-effective  methods  of  training,  compared  to  the  use  of  actual  equipment 
or,  in  the  case  of  computer-based  instruction,  conventional  classroom  instruction.  The  data 
needed  to  support  this  finding  are  robust  and  are  derived  from  about  KK)  studies  of 
simulators  and  hundreds  more  for  computer-based  instruction. 

Nevertheless,  it  is  important  to  understand  what  these  findings  do  and  do  not  mean. 
All  evaluations  of  cost-effectiveness  reported  here  are  based  on  data  for  individual  training 
in  institutions.  Further,  they  deal  with  initial  rather  than  advanced  training.  Demonstration 
of  their  cost-effectiveness  in  this  environment  does  not  necessarily  extend  to  advanced 
individual  training  or  to  collective  training  in  any  place. 

Even  if  a  simulator  is  equally  effective  in  all  environments,  the  costs  of  its  use  will 
vary  as  a  function  of  its  utilization  (i.e.,  hours  per  day),  the  support  provided  for  meaning¬ 
ful  training  (instructors  and  maintenance  personnel),  and  the  tasks  for  which  it  provides 
training.  The  conditions  of  unit  training  differ  markedly  from  those  at  a  school  with  respect 
to  full-time  availability  of  instructors  and  maintenance  personnel,  differing  priorities  for 
training,  and  the  attention  given  to,  or  even  the  existence  of,  training  plans  for  guiding  what 
is  supposed  to  be  learned.  It  is  difficult  to  believe  that  training,  even  if  effective,  can  be 
performed  as  efficiently  (i.e.,  at  the  same  cost)  in  units  as  in  schools.  Collective  training  at 
an  institution  for  personnel  assigned  to  a  unit  entails  the  costs  of  transporting  personnel  and 
equipment  from  the  unit  to  the  school  or  field  exercise  facility  and  of  their  support  while 
there,  as  well  as  the  costs  attributed  to  their  not  being  able  to  perform  their  regular  jobs 
while  away  from  the  unit.  These  considerations,  depending  on  how  they  weigh  in  the 
outcome  of  any  specific  case,  can  easily  change  the  cost-effectiveness  of  a  method  of 
training  as  its  venue  changes.  This  is  why  flight  simulators — expensive  devices — can  be 
cost-effective  for  training  at  institutions  but  not  in  units,  where  they  would  otherwise  be  a 
highly  useful  asset.  If  the  cost  of  simulators  could  be  reduced,  the  outcome  might  favor 
their  use  in  units.  This  appears  to  be  the  case  for  "unit  training  devices"  now  being 
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developed  by  the  Air  Force  (described  more  fully  in  Chapter  IV);  these  devices  are  less 
expensive  than  flight  simulators  and  are  computer-based,  part-task  trainers  that  can  be 
easily  modified  to  provide  several  different  types  of  training  in  units. 

In  contrast,  on-the-job  training  is  not  only  a  necessary  type  of  training  but  it  can  be 
performed  only  in  units.  Except  for  jobs  that  require  little  skill  (e.g.,  data  processing 
technician,  mess  management  specialist),  it  is  not,  however,  a  cost-effective  method  of 
training  because  the  newly  qualified  students  cannot  perform  their  assigned  tasks  well, 
create  errors,  require  supervision,  and  are  placed  in  an  environment  that  is  not  optimized 
for  training.  The  trend  to  "embedded  training,"  that  is,  the  use  of  training  features  built  into 
actual  equipment,  provides  simulated  targets  and  a  performance-recording  capability  that 
supports  individual  and  collective  training  in  operational  units. 

It  is  understandable,  but  not  necessarily  prudent,  to  favor  the  use  of  simulators  that 
train  as  well  as  actual  equipment  but  cost  less  to  procure  and  use.  At  issue,  fundamentally, 
is  the  purpose  of  using  simulators  for  training:  to  train  as  well  as  or  better  than  is  possible 
with  actual  equiment.  The  purpose  of  military  training — and  of  the  military  establishment 
itself— is  not  to  save  money  (or  to  make  a  profit).  Its  purpose  is,  clearly,  to  win  wars. 
Military  weapons  are  always  justified  in  terms  of  better  performance,  preferably  at  no 
increase  in  cost,  but  at  greater  cost  if  necessary.  The  same  paradigm  should  apply  to 
simulators  where  the  criterion,  at  least  for  military  application,  should  be  to  provide 
superior  performance  to  match  the  superior  performance  of  advanced  weapons.  The 
current  paradigm  that  supports  the  acquisition  of  simulators  that  train  as  well  as  actual 
equipment  at  less  cost  is  short-sighted.  We  should  change  the  paradigm  so  that  training 
matches  the  one  that  applies  to  military  weapons:  improved  performance  at  no  increase  in 
costs. 

Figure  III-7  shows  the  types  of  simulators  and  simulations  that  can  be  used  for 
individual  or  collective  training  in  institutions  or  in  units.  With  some  exceptions,  one  may 
observe  that  more  costly  simulators  tend  to  be  located  at  institutions  rather  than  in  units. 
This  is  a  way  of  insuring  more  utilization  and  technical  support  than  could  occur  in  units. 
The  reverse  is  also  true:  units  can  afford  to  use  (or  are  offered)  less  complex  and  less 
costly  simulators  than  are  used  in  institutions.  Two  current  trends  are  apparent:  reductions 
in  the  cost  of  electronic  equipment  make  it  increasingly  reasonable  to  use  certain  trainers  in 
units,  e.g.,  unit-training  devices  and  embedded  trainers.  The  other  trend  is  that  less 
expensive  computing  equipment  as  well  as  high-capacity  communications  can  now  support 
distributed  interactive  simulators  for  collective  training  in  units.  More  powerful  technology 
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at  reduced  cost  favors  the  use  of  advanced  simulators  for  collective  training  in  units  but 
cost-effectiveness  evaluations  to  support  this  application  remain  to  be  conducted. 

Where  Training  Occurs 

Institutions _ Units _ 

Type  of  on-the-job  training 

Training  flight  simulators  ‘  ^ 

maintenance  simulators  computer-based 

Individual  computer-based  instruction 


Collective 


ENWGS  Enhanced  Naval  War  Gaming  System 

CBS  Corps  Battle  Simulation 

ACAAM  Air  Courses  of  Action  Assessment  Model 

SIMNET  Simulator  Networking 

CCTT  Close  Combat  Tactical  Trainer 

STOW  Synthetic  Theater  of  War 

MDT2  Multi-Service  Distributed  Training  Testbed 

Figure  III-7.  Types  of  Simulators  and  Simulations  That  Tend  to  Be  Used  for 
Individual  or  Collective  Training  in  Institutions  or  in  Units 

FINDINGS 

1 .  Studies  that  have  examined  both  the  cost  and  effectiveness  of  simulators  for 
training  were  found  only  in  three  areas:  flight  simulators,  maintenance  simulators,  and 
computer-based  instruction.  The  utility,  i.e.,  the  cost-effectiveness,  of  these  devices  for 
training  is  supported  by  extensive  data.  When  students  are  tested  on  actual  equipment, 
training  with  these  simulators  is  as  effective  as  training  on  actual  equipment.  The  costs  of 
using  a  simulator  are  always  less  than  that  of  using  actual  equipment;  operating  costs  range 
from  about  8  to  50  percent  that  of  actual  equipment,  depending  on  the  type  of  simulator  and 
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actual  etjuipnient  being  exainined.  Acquisition  and  life  cycle  costs  of  simulators  are  about 
half  that  of  actual  equipment.  The  magnitude  of  annual  savings  permits  the  cost  of  these 
simulators  to  be  amortized  in  from  2  to  4  years. 

2.  While  simulators  are  demonstrably  a  cost-effective  method  of  training  for 
certain  tasks,  this  does  not  mean  that  actual  equipment  should  not  also  be  used  for  training. 
The  central  issue  is  to  determine  the  optimum  combination  of  simulators  and  actual 
equipment  for  training  on  the  basis  of  effectiveness  and  overall  cost.  This  will  vary,  of 
course,  by  type,  complexity,  cost  of  the  simulator,  and  the  tasks  for  which  the  simulator  is 
used  compared  to  the  use  of  actual  equipment.  Issues  of  this  type,  which  are  fundamental 
to  a  rational  choice  for  training,  have  barely  been  examined.  On  the  whole,  only  the  cost- 
effectiveness  of  simulators  for  initial  training  at  institutions  has  been  examined,  the  results 
favor  the  use  of  simulators  in  institutions.  This  type  of  evaluation  has  not  been  conducted 
for  the  use  of  simulators  in  all  other  types  of  training,  i.e.,  individual  training  in  units  and 
collective  training  in  institutions  and  in  units. 

3.  The  paradigm  used  to  select  simulators,  rather  than  actual  equipment  for 
training,  is  a  limited  one:  equal  effectiveness  at  less  cost.  That  is  contrary  to  the  rule 
generally  observed  for  new  weapons  and  advanced  technology  where  greater  effectiveness, 
even  at  greater  costs,  is  an  accepted  paradigm.  Greater  effectiveness  for  our  weapons  is 
always  preferred  and  is  a  premium  in  design,  in  order  to  counter  the  numerical  superiority 
of  an  enemy  and  to  reduce  American  casualties.  More  effective  weapons  generally  rely  on 
leading  edge  technology,  are  not  easy  to  achieve  and  hence  the  acceptance  of  greater  costs 
for  marginal,  but  critical  improvements.  High  technology  weapons  need  not  be  expected  to 
perform  at  their  superior  capabilities  unless  personnel  trained  to  operate  them  can  also 
perform  at  higher  levels  of  effectiveness.  Efforts  to  develop  and  to  use  simulators  for 
training  that  provides  increased  effectiveness,  with  little  or  no  increase  in  costs  is  a  clearly 
desirable,  although  generally  neglected,  goal  for  simulators  to  be  used  for  training. 


IV.  SERVICE  EXPERIENCE  WITH  SIMULATION 


This  chapter  describes  an  effort  to  identify  the  factors  that  may  account  for  the 
success  or  lack  of  success  in  the  use  of  simulations  by  the  Services.  All  Services  were 
approached  with  a  request  to  identify  examples  of  the  successful  application  of  simulations 
as  well  as  examples  in  which  problems  had  been  encountered.  The  purpose  of  this  effort, 
it  was  explained,  was  to  identify  features  of  successful  simulations  that  could  be 
incorporated  or  enhanced  in  future  developments  and,  in  the  case  of  problems,  to  suggest 
ways  to  improve  the  design  and  use  of  new  simulations.  Information  was  solicited  from 
the  human  resources  laboratories,  training  conunands,  and  simulation  and  training  device 
development  agencies.  Access  to  the  training  and  operational  conunands  was  helped  by  the 
participation  of  retired  officers  of  all  Services.  Sampling  was  opportunistic  rather  than 
rigorous;  with  few  exceptions,  those  contacted  tended  to  be  helpful  and  provided  useful 
information. 

We  were  able  to  identify  simulations  that  our  respondents  regarded  as  successful. 
No  "unsuccessful"  simulations  were  volunteered  or  could  be  elicited;  however,  various 
types  of  problems  were  found.  Below,  we  list  simulations  that  were  reported  by  each 
Service  as  either  "successful"  or  having  "problems."  Some  simulations  are  categorized  as 
"undetermined"  because  data  on  their  effectiveness  are  not  yet  available;  most  of  these  are 
current  development  programs  for  which  expectations  are  high. 

ARMY  SIMULATIONS! 

We  list  below  the  simulations  reported  by  Army  personnel. 


!  Most  of  the  information  in  this  section  was  gathered  by  Lt.  General  Frederic  J.  Brown,  USA  (Ret.). 
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Successful  Simulations 

Multiple  Integrated  Laser  Engagement  System  (MILES)^ 

Success  due  to  good  instrumentation,  supported  by  careful  development  of 
after-action  review  procedures,  and  use  of  trained  observer  controllers  at  the 
National  Training  Center 

Conduct  of  Fire  Trainer  (COFT)^ 

Development  of  COFT  was  preceded  by  identification  of  64  tasks,  conditions 
and  standards  determined  to  be  critical  to  crew  proficiency  in  Armored  Fighting 
Vehicles.  Success  attributed  to  development  of  a  training  program  with  a 
simulator,  rather  than  only  to  a  gunnery  trainer. 

National  Training  Center  (NTC)  and  other  Combat  Training  Centers  (CTC) 

Accurate  performance  measurement  provides  critical  information  that  improves 
training  effectiveness. 

Simulators  with  Problems 

Rifle  marksmanship  on  instrumented  range  (TRAINFIRE) 

Inadequate  management  of  funds  for  construction  and  allocation  of  terrain  for 
use  of  TRAINFIRE.  Provided  capability  for  shooting  at  an  approaching 
enemy  whereas  most  combat  targets  move  laterally. 

Improved  Tank  Training  Ammunition 

Thermal  and  light  blooming  effects  of  Service  ammunition  were  not  recognized 
in  time  because  safety  considerations  precluded  actual  training  with  an 
opposing  enemy  using  service  ammunition.  There  was  also  a  mismatch 
between  ammunition  capability  and  the  optical  devices  used  for  acquisition  and 
for  measuring  performance  in  training. 

Army  Fighting  Vehicle  Tactical  Tables 

Tactical  tables  were  an  innovation  in  mounted  close  combat  training  needed  by 
a  unit  to  operate  on  a  360°  battlefield.  The  new  tactical  tables  were  not 
established  or  supported  as  mandatory;  only  earlier  Gunnery  Tables  were 
trained  consistently. 


2  MILES,  COFT  and  SIMNET  were  used  to  support  training  for  the  U.S.  team  that  won  the  Canadian 
Armor  Trophy  in  1987. 

3  Training  the  crews  of  four  battalions  to  achieve  100  percent  certification  on  UCOFT  (Unit  Conduct  of 
Fire  Trainer)  before  live-fire  gunnery  improved  the  average  score  on  Table  Vin  from  756  to  837  (11 
percent)  (Blackwell  and  Brown,  1994). 


Reserve  Component  Training 

Basic  assumption  is  that  training  support  developed  for  the  Active  forces 
transfers  automatically  to  the  Reserve  component.  Different  training  methods 
and  doctrine  are  needed  for  the  reserves  where  time  for  training,  rather  than 
personnel  or  funds,  is  the  most  critical  resource. 

Undetermined 

Required  effectiveness  data  not  yet  available;  most  are  current  development 
programs 

Tactical  Engagement  Simulation  (TES) 

National  Guard  Virtual  Training  Program 
Battle  Command  Staff  Training  (BCST) 

73  Easting,  used  interactively  for  "what-if '  exploration 
Modular  Semi-automated  Forces  (ModSAF) 

Rapid  Task  Training-prior  to  deployment 

Adaptation  of  Tactical  Engagement  Simulation  training  strategies  for  other 
nations  and  for  less  capable  U.S.  personnel 

All  successfiil  simulations  have  high  user  acceptance.  They  also  have  the  benefit  of 
a  training  program  that  shows  how  use  of  the  simulator  meets  an  accepted  requirement. 
This  is  in  direct  contrast  to  less  successful  simulations  that  do  not  integrate  use  of  the 
simulator  into  the  overall  training  program.  Some  less  successful  simulators  had  specific 
technical  deficiencies,  such  as  the  inadequate  terrain  data  base  in  TRAINFIRE  (due  to 
insufficient  funding),  or  blooming  effects  of  tank  training  ammunition  on  thermal  sights; 
such  cases  suggest  either  unanticipated  technical  problems  or  insufficient  test  and 
evaluation  during  the  developmental  process.  The  technical  deficiencies,  such  as  they  are, 
do  not  appear  to  be  challenging  or  insurmountable.  One  suspects  that  the  attractiveness  of 
new  technology  and  insufficient  concern  with  how  it  should  be  used  in  the  training 
program  can  account  for  most  deficiencies  in  this  sample  of  Army  simulations. 

Lt.  General  Frederic  J.  Brown,  USA  (Retired)  (1993,  p.  5-1)  makes  the  point  that: 

the  combination  of  the  training  requirements  mandated  by  both  AirLand 
Battle  and  the  constraints  generated  by  the  ever-present  and  growing 
restrictions  on  conducting  field  training  (funds,  terrain,  ecology,  soldier 
time,  and  ever-present  safety)  can  be  attained  only  by  deliberate  design  or 
"structuring"  of  the  training  process  to  ensure  that  specific  training  events 
occur  in  the  manner  and  sequence  desired  to  achieve  intended  task  training 
purposes.  This  structuring  seems  essential  if  current  training  practices  are 


to  be  retained.  Perhaps  more  important,  it  provides  a  way  to  train  for  the 

future,  to  train  Third  Wave  Landpower _ It  became  evident  that  current 

training  doctrine  does  not  establish  a  rationale  or  methodology  for  rigorous 
design  of  explicit  task  training  to  standard  advantaging  emerging  Tactical 
Engagement  Simulation  (TES).  Until  training  development— a  statement  of 

explicit  training  purposes  and  associated  rules  for  use— is  established,  the 

promise  of  training  support  (TES)  cannot  be  realized. 

NAVY  SIMULATIONS^ 

The  following  simulations  were  reported  by  the  Navy. 

Successful  Simulations 
SH-60B 

Operational  Flight  Trainer  (OFT) 

P-3C 

Operational  Flight  Trainer  (OFT) 

Weapon  System  Trainer  (WST) 

F/A-18  OFT,  WTT 

Tactical  Air  Combat  Training  System  (TACTS) 

Battle  Force  Tactical  Training  (BFTT) 

Simulations  with  Problems 

Enhanced  Naval  Warfare  Gaming  System  (ENWGS) 

Undetermined  (under  development) 

Maritime  Synthetic  Theater  of  Warfare  (MSTW) 

Aviation  flight  and  tactical  simulators  are  reported  to  be  particularly  successful  in 
terms  of  transfer  of  training  with  appreciable  savings  in  overaU  cost,  although  documenta¬ 
tion  of  these  findings  was  not  provided.  Helicopter  and  maritime  patrol  aircraft  simulators 
are  considered  by  many  operators  to  be  better  candidates  than  other  aircraft  for  simulator 
hour  substitution  because  they  do  not  require  high  g  loads  that  fighter  pilots  say  is 


^  The  following  were  particularly  helpful  in  providing  information  about  the  Navy’s  efforts  in 
simulation:  David  Glenn,  NAWC/TSD;  Andrew  Appleget,  NUWC:  Rudy  Croteau,  NUWC;  Jerry 
Lema,  Naval  War  College;  Peter  Kincaid,  Institute  for  Simulation  and  Training;  CDR  Michael 
Lilienthal,  DMSO;  Dianne  Dry,  IDA  Simulation  Center;  CDR  James  Clagor,  (N889-F2)  Training 
Device/NTP  Coordinator,  Aviation  Readiness;  CAPT  Tom  Travis  (N812D),  Head,  Modeling  and 
Analysis;  George  Phillips  (N812D2),  Team  Mike;  CAPT  Raymond  Morris,  PMA-205;  Louis 
Solomon,  ARPA. 
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important  in  a  flight  simulator.  Additionally,  for  SONAR,  RADAR,  ECM^  and  MAD^ 
operators  and  tactical  coordinators,  the  working  environment  of  the  simulator  is  little 
different  from  that  experienced  in  flight.  The  primary  test  of  a  successful  simulator  for 
training  these  crew  members  is  the  fidelity  of  the  simulated  target  signals  and  equipment 
displays. 

User  acceptance  of  flight  simulators  is  generally  high.  The  Tactical  Air  Combat 
Training  System  (TACTS)  ranges  have  been  particularly  successful  in  providing  Air 
Combat  Maneuvering  (ACM)  training  and  are  an  integral  part  of  the  Navy  and  Marine  inter¬ 
deployment  training  programs.  The  Battle  Force  Tactical  Training  (BFTT)  system, 
although  still  under  development,  incorporates  components  that  have  demonstrated 
successful  transfer  of  training  into  a  full  DIS-capable  training  system  for  individual  through 
Joint  training  exercises  (again,  without  documentation). 

The  Enhanced  Naval  War  Gaming  System  (ENWGS)  is  considered  less  than 
successful  primarily  in  its  role  in  supporting  the  Naval  War  College  War  Gaming  Center. 
ENWGS  is  a  command-level  exercise  trainer,  dependent  on  computer-based  combat 
models,  i.e.,  a  simulation  rather  than  a  simulator.  Although  it  may  be  useful  in  some 
applications,  it  has  not  been  adapted  to  the  requirements  for  support  of  high-level  training 
games. 

Flight  Simulators 

Operational  Flight  Trainers  (OFTs),  Weapons  System  Trainers  (WSTs),  and 
Weapons  Tactical  Trainers  (WTTs)  as  a  class  have  been  some  of  the  most  successful 
trainers  in  the  Navy  inventory.  Helicopter  trainers  have  been  particularly  successful 
because  helicopters  do  not  experience  high  g  forces,  and  because  many  emergencies  cannot 
be  practiced  safely  in  the  aircraft.  Even  some  basic  maneuvers,  such  as  landing  on  a  small 
ship,  require  extremely  precise  control  and  are  best  introduced  in  a  simulator.  Simulators 
are  also  well  suited  for  training  non-pilot  crew  members  of  both  helicopters  and  maritime 
patrol  aircraft.  The  flight  environment  for  these  air-crew  is  little  different  from  that 
experienced  in  the  simulator.  Except  for  differences  in  noise  and  vibration  levels,  which 
can  be  simulated  for  cosmetic  reasons,  stimuli  external  to  the  crew  compartment  are 
essentially  absent. 


5  ECM:  Electrcmic  Countermeasures 
®  MAD:  Magnetic  Anomaly  Detector 
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The  F/A-18  OFT  and  WTT  are  reasonably  representative  of  the  current  technology 
of  aviation  simulators  and  are  an  integral  part  of  the  Fleet  Replacement  Pilot  training 
program.  The  Category  1  (Fleet  Replacement  Squadron)  training  syllabus  includes  40 
trainer  and  77  aircraft  flights.'^  A  1991  Naval  Training  Systems  Center  (NTSC)  report 
showed  that  the  WTT  was  effective  in  training  transfer  for  both  air-to-air  and  air-to-ground 
maneuvers  (Pfeiffer  and  Dwyer,  1991).  The  subjects  of  this  study  were  Fleet  Replacement 
Pilots  and  the  results  may  not  be  extrapolated  directly  to  fleet  pilots  who  are  more 
advanced.  A  1992  IDA  study  showed  that  transfer  of  training  from  simulators  to  aircraft 
does  occur  for  F/A-18  fleet  pilots  for  the  air-to-ground  mission  (Hammon  and  Horowitz, 
1992).  Even  so,  some  fleet  pilots  stated  that  simulators  are  not  useful  for  continuation 
training  because  simulator  flights  do  not  produce  the  requisite  rush  of  adrenaline  and  the 
simulation  of  g  forces  is  not  realistic.  This  opinion  obviously  reduces  user  acceptance  even 
though  objective  studies  show  that  such  simulators  are  effective  training  devices.  A 
summary  of  the  reasons  for  successful  simulators  in  the  Navy  is  given  in  Fig.  IV- 1 . 

The  alternative  to  using  simulators  is  to  do  all  training  in  the  aircraft.  This  is  not 
only  more  expensive,  but  may  be  dangerous.  For  one  maneuver,  Pfeiffer  and  Dwyer, 
1991,  could  not  use  an  "aircraft  only"  control  group  because  it  was  not  considered  safe  to 
start  Fleet  Replacement  Pilots  out  in  a  single  place  aircraft,  and  not  enough  two  place 
aircraft  were  available.  Additionally,  many  helicopter  emergencies  cannot  be  simulated  in 
the  aircraft  for  safety  reasons. 

Flight  simulator  technology  basically  uses  off-the-shelf  aircraft  software  and  instm- 
mentation.  Simulator  technology  is  well  developed,  but  modifications  to  meet  standard 
protocols  for  networking  are  needed  to  advance  aircraft  tactical  simulation  into  the  DIS 
mode  and  to  attain  needed  portability. 

Although  acceptance  of  the  OFT  and  WTT  by  Fleet  Replacement  Squadrons  is  quite 
high,  usage  data  for  fleet  squadrons  indicate  that  further  substitution  by  fleet  squadrons  is 
possible.  A  sample  of  Marine  squadrons  used  these  trainers  for  an  average  of  0.3  hours/ 
pilot/month  in  FY92.  The  average  for  all  east  coast  Navy  F/A-18  squadrons  was 
1.5  hours/pilot/month.  Average  flying  hours/pilot/month  were  21  and  18.5,  respectively. 


^  VMFAT  101  Training  Syllabus. 
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Reason  for  Success 

Simulator 

high  user  acceptance 

ail  cases 

acceptable  fidelity 
target  signals 
air  combat  maneuvering 
visual  display 
g  forces 

P-3C,SH-60B 

TACTS,  F/A-18  WTT 
dome  technology  for  F/A-18 
bladder,  g  suit  F/A-18 
g  forces  not  needed  for  P-3C,  SH-60B 
because  aircraft  maneuver  motion  at 
low  g  forces 

combat  work  environment 

for  new  pilots  in  F/A-18 

train  emergency  procedures 

BFTT,  P-3C,  SH-60B,  F/A-18 

provides  feedback  for  training 

all  cases 

cost  savings 

BFTT  saves  OPTEMPO 

F/A-18  flying  hours  reduced  0.3-1 .5  hrs 
per  month 

no  acceptable  alternative  for 
training 

4-6 

TACTS 

Figure  IV-1.  Reasons  for  Successful  Simulations  in  the  Navy 

F/A-18  simulators  show  some  shortfalls  with  regard  to  their  ability  to  simulate  the 
full  mission  environment  and  transportability  to  aircraft  carriers.  Positioning  the  aircraft  for 
an  air-to-ground  weapon  delivery  is  a  demanding  task,  but  the  true  measure  of  mission 
mastery  is  to  be  able  to  get  to  the  target,  deliver  one's  weapons  effectively  and  egress  safely 
in  a  hostile  environment.  Current  simulators  do  not  test  the  air-crew's  ability  in  an 
environment  that  includes  ground  opposition  and  electronic  warfare.  In  the  past,  these 
tasks  could  be  practiced  to  some  extent  in  the  aircraft.  However,  as  budgets  decrease,  we 
have  an  increased  requirement  for  coordinated  operations  in  a  simulated  hostile 
environment.  Additionally,  transportability  is  needed  so  that  simulators  can  be  used  aboard 
aircraft  carriers  to  review  basic  flight  and  emergency  procedures,  and  for  mission  rehearsal. 

WTTs  are  currently  installed  in  suites  of  two  and  have  a  maximum  interoperability 
of  one-on-one  or  two  aircraft  against  up  to  four  simulated  targets.  This,  along  with  the 
inability  to  introduce  an  electronic  warfare  environment  or  ground  fire,  limits  the  potential 
usefulness  of  the  device,  especially  for  fleet  squadrons.  Mission  rehearsal  capability  is  also 


limited  by  the  lack  of  recording  and  retrieval  of  performance  data,  and  by  the  size  of  the 
simulator  installation.  Operating  and  Support  costs  per  hour  for  the  F/A-18  WTT  are 
approximately  $250,  just  over  10  percent  of  the  cost  of  one  hour  of  F/A-18  flying  time. 

The  Air  Force  is  examining  the  utility  of  distributed  interactive  simulation  to  provide 
air  combat  training  by  using  the  multi-ship  air  combat  simulator  facility  for  the  F-15E  at 
McDonnell  Aircraft  Company  (Thomas  and  Houck,  1990).  The  Air  Force  and  Navy  are 
also  examining  simulator  networks  in  conjunction  with  the  Joint  ACM  Training  System. 
The  Naval  Aviation  Training  Systems  Advisory  Group  (NATSAG)  has  made  this  a 
Technology  Requirement  action  item.®  Currently  the  Navy  has  no  technology  requirement 
for  developing  a  multiple  aircraft,  electronic  warfare,  ground  threat  environment  for 
We^ons  Tactical  Trainers.  This  might  be  a  potentially  high  pay-off  technology  area. 

Portability  might  be  attained  through  the  development  of  virtual  environment  (VE) 
technology.  A  VE  trainer,  with  a  helmet  a  little  heavier  than  a  standard  flight  helmet,  would 
fit  in  a  cabinet  which  is  approximately  10x6x3  feet.  An  F/A-18  trainer  test  bed  which 
has  received  quite  high  marks  from  F/A-18  pilots  is  being  developed  at  the  Naval  Air 
Warfare  Training  Center.  At  this  point,  its  major  deficiency  is  that  the  field  of  vision  does 
not  respond  to  head  movements.  However  NAWC  personnel  are  confident  that  this  can  be 
overcome.  They  cite  work  with  a  liquid  crystal  display  (LCD)  projection  system  develop¬ 
ment  at  MIT  as  a  performance  requirement  and  research  by  the  entertainment  industry  as 
indicators  that  the  program  can  be  successful.  The  Institute  for  Simulation  and  Training  at 
the  University  of  Central  Florida  has  developed  a  head  mounted  display  which  does  keep 
track  of  head  position  and  where  the  subject  is  looking. 

Tactical  Air  Combat  Training  Systems  (TACTS) 

The  Tactical  Air  Combat  Training  System  (TACTS)  ranges  are  used  to  train  fleet 
pilots  in  intermediate  and  advanced  air-to-air  combat  skills.  The  ground  tracking  system 
keeps  track  of  all  aircraft  on  the  range  and  records  simulated  attacks  and  results.  The  total 
tactical  picture  is  recorded  for  later  debrief  of  the  participants. 

The  only  current  alternative  to  training  on  the  TACTS  ranges  is  uninstrumented  air- 
to-air  exercises.  Such  engagements  are  practiced  from  day  to  day  as  part  of  the  squadron 
training  syllabus,  but  actual  outcomes  and  information  on  the  use  of  proper  procedures  are 


®  CNO  (N889)  Itr.  ser.  889F4/3U658330  of  10  December  1993,  Subj:  NATSAG  Conference  13-15 
October  1993. 


speculative.  Readiness  for  flight  on  a  TACTS  range  is  provided  by  WTT  and  WST 
simulator  flights.  However,  these  simulators  do  not  approach  the  TACTS  ranges  in 
realism  of  threat  environment,  numbers  of  units,  or  difficulty. 

With  the  stand-down  of  two  of  the  five  active  duty  adversary  squadrons,  the  Navy 
recognizes  a  need  for  alternative  ways  to  exercise  against  dissimilar  aircraft  and  air-crews 
outside  the  immediate  squadrons.  There  is  a  further  and  more  critical  need  to  incorporate 
TACTS  range  fleet  exercises  with  the  full  carrier  battle  group  and  with  joint  forces. 
Advanced  exercises  and  research  and  development  also  require  secure  links  to  the  aircraft  to 
ensure  that  the  data  cannot  be  exploited  for  intelligence  purposes,  by  either  a  real  enemy  or 
one  in  an  exercise.  A  long  recognized  shortfall  of  the  system  is  the  inability  to  sort  out 
targets  or  friendly  aircraft  that  are  within  the  same  spatial  cone  but  at  different  ranges. 

Much  of  the  potential  improvement  of  TACTS  requires  no  real  stretch  in 
technology.  Rather,  the  requirement  is  for  cooperation  among  Navy,  Marine  Corps,  and 
Air  Force.  This  cooperation  has  been  initiated  with  agreement  to  the  operational  concepts 
required  of  the  TACTS/ACMI  system  and  JACTS  (Joint  ACTS).  These  programs  would 
require  further  development  of  technology  in  order  to  upgrade  the  electronic  warfare  threat 
environment,  encrypt  the  computer/aircraft  links,  and  resolve  the  target  resolution  problem. 
These  technologies  are  well  within  reach  with  the  possible  exception  of  target  resolution 
which  may  be  difficult  to  achieve.  Problems  with  some  simulators  in  the  Navy  are 
summarized  in  Fig.  IV-2.  Desired  improvements  in  some  Navy  simulations  are  listed  in 


Fig.  IV-3. 


lack  of  jamming 

TACTS 

lack  of  ground  threats 
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need  multiple  friendly  and  enemy  aircraft 

F/A-18 

lack  of  submarines,  aircraft 

BFTT 

simulator  expensive  and  not  portable  (for 
carrier  use) 

all  OFT,  WST,  WTT 

limited  performance  recording  and 
retrieval  for  feedback 

it 

limited  mission  rehearsai  capability 

a 

lack  encrypted  links  to  other  platforms 
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Figure  iV-2.  Problems  With  Some  Simulators  in  the  Navy 


Desired  Imorovement 

Simulator 

simulation  of  more  combat  units 

BFTT 

include  aircraft  simulators 

BFTT 

include  USMC  landing  force 

BFTT 

central  scenario  generator 

BFTT 

lightweight  headset  and  see-through 
projection  surface 

Virtual  Environment  Testbed  at 
Naval  Air  Warfare  Center  Training 
Systems 

replace  high  cost  dome  visual  display 

F/A-18,  SH-60B  OFT/WTT 

semi-automated  force 

ENWGS 

compatability  with  distributed 
interactive  simulation 

ENWGS 

encrypted  links  to  all  platforms 

TACTS 

4-B 

Figure  IV-3.  Desired  improvements  in  Some  Navy  Simuiations 

The  Navy  is  currently  developing  a  technology  requirement  for  incorporating 
TACTS  into  a  fleet-wide  system  as  part  of  the  Tactical  Combat  Training  System  (TCTS). 
This  will  require  a  standardized  DIS  protocol,  as  well  as  an  expanded  geographical  area  of 
operations.  Additionally,  a  requirement  exists  for  a  simulation  environment  generator  that 
can  communicate  with  other  distributed  systems. 

Battle  Force  Tactical  Training  (BFTT) 

Battle  Force  Tactical  Training  (BFTT)  responds  to  the  draw-down  in  the  Naval 
force  and  OPTEMPO  and  is  intended  to  provide  individual  through  Joint  training 
throughout  the  training  cycle  while  ships  remain  in  their  home  port.  Ships  are  home 
ported  by  type  at  widely  separated  localities.  During  the  initial  and  intermediate  training 
phases,  it  is  not  practical  to  assemble  all  ships  of  a  Carrier  Battle  Group  in  the  same 
location.  BFTT  is  also  intended  to  take  advantage  of  the  assumed  superior  training 
environment  of  the  trainees'  own  ship. 
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The  BFTT  concept  has  received  high  user  acceptance  by  the  surface  community  and 
will  provide  the  on-board  capability  for  the  Tactical  Combat  Training  System  (TCTS).  The 
essential  elements  are  a  central  environment  generator  ashore,  a  communications  link,  and 
embedded  on-board  trainers  (OBTs).  This  system  has  been  demonstrated  using  equipment 
and  technology  being  developed  at  the  Navy  Undersea  Weapons  Center  at  New  London, 
Conn.,  and  Newport,  R.I.  arvine  and  Quintana,  1994).  These  are  the  Tactical  Advanced 
Simulated  Warfare  Integrated  Trainer  (TASWIT)  as  the  central  environment  generator,  the 
SSQ-89  sonar  OBT,  and  a  DIS-compatible  Local  Area  Network  (LAN). 

An  alternative  to  BFTT  is  to  use  on-shore  trainers  for  initial  individual,  team,  and 
unit  trainers  followed  by  operations  at  sea.  This  is  the  current  system  and  it  entails  moving 
personnel  to  existing  trainers  and  expensive  at-sea  operating  periods.  Individual,  team,  and 
unit  training  can  be  conducted  throughout  the  interdeployment  training  cycle  without  the 
requirement  for  TEMADD  funds  or  the  loss  of  personnel  for  other  shipboard  requirements. 
An  additional  advantage  is  that  BFTT  will  generate  real  time  data  that  can  be  used  for 
instantaneous  play-back  and  debriefing. 

The  technology  requited  to  complete  BFTT  exists  but  further  engineering  develop¬ 
ment  is  needed.  The  BFTT  program  manager  is  currently  pulling  together  existing  systems, 
such  as  TASWIT— or  an  alternative— and  existing  OBTs.  Many  existing  on-shore  trainers 
use  simulation  rather  than  stimulation.  Stimulation  is  a  term  used  in  the  Navy  when 
external  signals  are  fed  into  actual  operational  equipment,  such  as  radars  or  sonars,  to 
simulate  enemy  or  friendly  platforms.  Adding  a  stimulation  capability  for  all  equipment 
may  requite  additional  technology  advances.  Communications  links  compatible  with  DIS 
standard  protocol  are  available  and  technology  should  present  no  problem.  All  of  the 
candidates  for  the  BFTT  central  environment  generators  are  not  DIS  compatible  but  this  is 
not  necessarily  a  technology  problem. 

When  fully  operational,  the  goal  of  BFTT  is  to  provide  a  system  for  team  and  unit 
training  in  all  warfare  areas,  links  to  support  multi-ship  and  multi-port  in-port  training,  and 
the  gateway  for  the  fleet  to  the  Tactical  Combat  Training  System  (TCTS).  Both  the  shore- 
and  ship-based  equipment  required  for  TCTS  are  to  be  developed  as  part  of  BFTT.  For 
submarines,  however,  embedded  training  systems  are  in  the  very  early  stage  of  develop¬ 
ment  and  organic  data  collection  systems  do  not  yet  exist.  It  is  planned  to  initially  move 
submarine  personnel  ashore  with  full  ship-based  systems  to  follow  when  OBTs  and 
debriefing  systems  ate  fully  developed.  This  is  stated  to  be  an  engineering  rather  than  a 
technology  problem  by  personnel  at  Naval  Undersea  Warfare  Center  and  the  Naval  Air 
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Warfare  Center  Training  Systems  Division  who  are  responsible  for  integrating  the 
submarine  training  system. 

There  is  no  stated  requirement  for  incorporating  networked  aircraft  simulators  in 
BFTT.  This  seems  to  be  a  major  shortfall  in  that  controlling  aircraft  is  a  major  part  of 
carrier  battle  force  operations.  Additionally,  no  requirement  exists  for  surface  units  to  be 
able  to  interact  with  actual  aircraft  assigned  to  the  same  Carrier  Battle  Group  (CBG).  This 
will  further  limit  coordinated  training  opportunities.  Neither  will  BFTT  be  tied  in  with  a 
land-based  distributed  simulation  such  as  the  Marine  Air-Ground  Tactical  Force  (MAGTF) 
Tactical  Weapons  System  (MTWS). 

Another  potential  shortfall  arises  from  the  fact  that  BFTT  and  TCTS  are  supposed 
to  have  compatible  central  scenario  generators  but  not  necessarily  the  same  one.  This  could 
result  in  costly  redundancy  since  the  scenario  generator  will  be  an  integral  part  of  the  TCTS 
which  will  evenmally  reside  in  one  of  the  BFTT  ports  on  each  coast. 

Technology  issues  include  stimulation  of  all  equipments,  incorporation  of 
networked  aircraft  simulators  or  actual  aircraft  into  a  hostile  synthetic  EW  environment, 
incorporation  of  the  total  amphibious  task  force,  and  a  fully  DIS-capable  scenario 
generator.  Stimulation  of  all  equipments  will  require  more  advanced  technology  as  would 
networked  aircraft  or  aircraft  simulators.  Actual  aircraft  interoperability  would  entail 
technology  at  least  as  advanced  as  that  required  for  JACTS. 

Incorporation  of  the  amphibious  landing  force  in  BFTT  would  require  a  change  in 
synthetic  environment  generation  technology  beyond  that  planned  for  the  system.  As 
planned,  the  system  concentrates  on  the  ocean  and  air  environments.  Extending  this  to  a 
simulated  land  environment  would  be  difficult  to  achieve.  Even  without  dynamic 
modeling,  which  is  probably  well  in  the  future,  the  selection  of  a  central  scenario  generator 
for  BFTT  should  reflect  a  future  need  for  state-of-the-art  terrain  modeling.  The  Navy  is  in 
the  process  of  selecting  a  standard  scenario  generator  from  among  BFTT,  TCTS, 
TASWTT,  and  ENWGS.  Promising  synthetic  environment  generation  and  C3I  technology 
should  be  explored  in  conjunction  with  the  selection  of  a  central  scenario  generator  for 
BFTT. 

The  Enhanced  Naval  Wargaming  System  (ENWGS)  technology  uses  a  central 
processor  and  scenario  generator  which  supports  training  at  TACTRALANT, 
TACTRAPAC,  and  the  Naval  War  College.  ENWGS  is  also  one  of  the  final  four 
candidates  for  the  Navy  Exercise  Scenario  Generator.  The  remaining  text  will  concentrate 
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on  the  role  of  ENWGS  in  support  of  the  Naval  War  College  War  Gaming  Center  (NWC 
WGC).  The  NWC  WGC  is  responsible  for  supporting  (1)  educational  war  games,  (2)  war 
game  instruction  and  research,  and  (3)  high-level  strategic  games  for  defense  agencies 
including  U.S.  and  foreign  war  colleges.  Unified  and  Specified  commands,  and  the 
Department  of  the  Navy. 

Whereas  the  emphasis  of  NWC  WGC  is  on  high-level  strategic  games,  ENWGS  is 
targeted  at  the  tactical  level.  Of  55  games,  conferences,  and  seminars  conducted  by  the 
NWC  WGC  in  1992,  only  seven  used  ENWGS  for  scenario  generation.  According  to 
NWC  personnel,  ENWGS  is  too  detailed  and  not  capable  of  operating  at  big  enough  time- 
steps  for  their  purposes.  ENWGS  is  not  DIS  capable  although  a  program  is  under  way  to 
rehost  ENWGS  on  an  open  system  (JMCIS  architecture). 

The  NWC  is  investigating  the  possibility  of  computerizing  some  high-level  strategic 
games,  but  has  concluded  that  ENWGS  would  not  be  compatible  with  their  objectives. 
Besides  the  time-step  problem,  they  believe  that  it  is  important  for  their  customers  to  be  in 
their  own  surroundings  as  much  as  possible.  Seminar  type  games  better  meet  their 
educational  objectives.  ENWGS  is  used  for  some  war  college  student  games  at  the  tactical 
level. 

Enhanced  Naval  Warfare  Gaming  System  (ENWGS) 

ENWGS  technology  is  mature  computer  scenario  generation  and  retrieval  with 
man-in-the-loop  technology.  There  appear  to  be  no  major  development  problems 
remaining  except  to  adapt  the  system  to  DIS  compatible  open  architecture.  The  potential 
use  of  ENWGS  for  strategic  war  games  appears  to  be  limited.  This  does  not  appear  to  be  a 
technology  issue  as  much  as  a  budgetary  one.  The  system  appears  to  be  performing 
adequately  for  tactical  games  at  TACTRALANT  and  TACTRAPAC.  With  the  current 
rehosting,  ENWGS  will  be  supportable  and  not  proprietary  to  Honeywell  as  was  the  case 
until  recently. 

The  level  of  detail  and  data  base  generation  at  all  stations  in  ENWGS  contributes  to 
the  time-step  problem  cited  above.  There  are  no  plans  to  incorporate  Semi-Automated 
Forces,  which  is  a  critical  requirement  of  the  NWC  WGC.  This  is  related  to  the  level  of 
detail  and  time-step  problem.  The  system  is  not  DIS  compatible,  but  this  is  planned  for  the 
near  future. 

The  underlying  shortfall  appears  to  be  primarily  related  to  a  lack  of  user  require¬ 
ments  definition  when  the  system  was  initially  installed.  ENWGS  was  designed  for 
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TACTRALANT  and  TACTRAPAC  and  the  upgraded  version  was  also  responsive  to  these 
users.  Personnel  at  NWC  WGC  stated  that  there  was  very  little  interaction  between  them 
and  the  program  manager  and  the  system  just  grew. 

Department  of  the  Navy  and  Joint  Interoperability 

The  Navy  would  benefit  if  it  had  a  wargaming  system  that  could  respond  to  all 
potential  users.  Characteristics  which  should  be  included  are:  (1)  meets  requirements  for 
DIS  and  joint  interoperability,  (2)  leverage  off  the  MAGTF  Maritime  Warfare  System 
(MTWS)  and  Warfighters  Simulation  (WARSIM)  for  the  land  system  to  attain  Navy  and 
Army  compatibility,  (3)  leverage  off  the  National  Air  and  Space  Model  for  air  capability, 
and  develop  a  new  maritime  simulator.  This  same  approach  could  be  taken  by  exploiting 
the  technology  proven  by  ARPA  during  the  proof  of  concept  Maritime  Synthetic  Theater  of 
War/C3I  demonstration  conducted  in  September  1993.  This  demonstration  modeled  the 
sonar  and  mine  warfare  characteristics  of  a  portion  of  the  Sea  of  Japan.  The  system 
developed  by  ARPA  is  a  central  scenario  generator  that  relies  on  advanced  but  nearly 
proven  technology:  it  uses  advanced  computer  technology  and  a  physics  approach  to 
modeling. 

A  potential  Department  of  the  Navy  application  is  the  single  Navy  Scenario 
generator  with  a  full  DIS  and  joint  interoperability  capability.  By  including  all  Services  a 
scenario  generator  would  meet  the  maritime  needs  of  the  Services  and  of  joint  inter¬ 
operability.  An  initial  application  would  be  the  Battle  Force  Tactical  Trainer;  ARPA 
personnel  believe  this  would  be  a  low-risk  approach. 

The  probability  of  success  of  this  effort  is  estimated  as  high  by  the  ARPA  program 
office.  They  base  their  estimate  on  the  success  of  the  Maritime  Synthetic  Theater  of  War/- 
C3I  demonstration  and  the  fact  that  the  system  uses  a  physics  approach  to  scenario 
generation.  They  state  that  the  granularity  of  the  synthetic  virtual  scenario  is  close  to  our 
knowledge  of  the  physics  of  the  ocean,  land,  and  atmosphere.  Work  is  currently 
progressing  on  modeling  the  atmosphere — including  changes  in  weather  phenomena — for 
RADAR  applications.  Because  the  system  uses  advanced  computing,  ARPA  also  believes 
that  land  surfaces  can  be  modeled,  perhaps  as  a  dynamic  model.  The  major  shortcoming  at 
this  point  is  that  the  demonstration  model  does  not  include  a  sophisticated  recording  and 
debrief  capability.  This  capability  is  expected  to  be  a  matter  of  programming. 
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Summary 

Successful  simulations  in  the  Navy  are  those  that  have  high  user  acceptance, 
acceptable  levels  of  fidelity  (generally  because  the  simulator  closely  resembles  the  actual 
command  and  control  workplace,  such  as  in  the  P-3C,  BFTT  and  TACTS),  provide 
feedback  needed  for  training,  and  provide  some  cost  savings  (BFTT).  Simulators  that  have 
problems  tend  to  have  limited  capability,  e.g.,  incorporate  insufficient  numbers  and 
different  types  of  platforms;  this  deficiency  was  not  apparent  when  the  simulators  were 
designed.  The  desired  improvements  seem,  in  all  cases,  to  be  needed  to  enlarge  capability. 
Since  current  technology  appears  able  to  meet  most  of  these  expressed  needs,  the  real  issue 
is  probably  one  of  priority  in  allocating  funds  and  not  one  of  technical  feasibility.  We 
found  no  reports  that  a  simulator  had  problems  because  it  did  not  fit  into  a  well-developed 
training  program,  as  was  observed  for  some  Army  simulations. 

MARINE  CORPS  SIMULATIONS^ 

The  Marine  Corps  training  program  depends  heavily  on  the  use  of  simulators. 
About  90  percent  of  the  Marine  Corps  investment  in  simulators  for  training  is  allocated  to 
flight  training.  Successful  applications  are  reported  in  five  functional  areas;  brief 
descriptions  of  the  training  systems  follow. 


^  We  acknowledge,  with  appreciation,  assistance  provided  by  the  following  individuals: 

Marine  Corps  Modeling  and  Simulation  Management  Office  (MCMSMO), 

Marine  Corps  Combat  Development  Command  (MCCDC):  Lt  Col  John  Boyd,  USMC,  Mr.  Gary 
Brisbois,  and  Colonel  John  Kline,  USMC. 

Training  and  EducaUon  (T&E)  Division,  MCCDC:  Major  Doug  Doerr,  USMC,  Major  Ed  Flores, 
USMC,  Major  Chad  Kirkley,  USMC,  Colonel  Mike  Monigan,  USMC,  LtCol  Robert  Orazi,  USMC, 
and  Colonel  Walter  Wood,  USMC. 

MAGTF  Staff  Training  Program  (MSTP)  Division,  MCCDC:  Major  Floyd  Johnson,  USMC,  Mr. 
Dick  Morrell,  LtCol  Paul  Roy,  USMC,  Mr.  Jeff  Tkacheff,  and  C^tain  David  Wood,  USMC. 

Marine  Corps  Systems  Command  (MarCorSysCom):  Colonel  John  Cummings,  USMC,  Colonel  Ki 
Harvey,  USMC,  LtCol  Mike  Przepiora,  USMC,  and  Major  Frank  Wysocki,  USMC. 

Naval  Air  Warfare  Center,  Training  Systems  Divison  (NAWC/TSD):  Dr.  Charles  Beagles,  Mr.  Rick 
Beger,  Dr.  Robert  Hayes,  Major  Bruce  Henry,  USMC,  Major  Joe  Santilli,  USMC,  MSgt  Will  Tucker, 
USMC,  Colonel  Robert  Tyler,  USMC,  and  Mr.  Larry  Willis. 

Marine  Corps  Headquarters  (HQMC):  Mrs.  Linda  Goodwin. 


Marksmanship  and  Target  Engagement 

Indoor  Simulated  Marksmanship  Trainer  (ISMT) 

Extrapolation  from  an  extensive  operational  test  program  indicates  that  ISMT  is  a 
cost-effective  complement  to  live-fire  training  programs:  Weapon-qualification  rates 
improve  (especially  for  women  Marines);  ammunition  costs  are  reduced;  less  remedial 
training  is  needed  for  weapon  qualification;  and  fewer  recruits  are  separated  for  failure  to 
qualify  with  the  rifle. 

Precision  Gunnery  Training  System  (PGTS) 

TOW^o  gunners  trained  initially  with  PGTS-TOW  in  Southwest  Asia  during  Desert 
Shield  said  it  was  "just  like  the  real  thing."  In  Desert  Storm,  TOW  crews  achieved  high 
rates  of  hits  and/or  kills  per  missile  fired;  one  crew  destroyed  10  tanks  or  armored  vehicles 
with  10  missiles.  Similar  success  with  PGTS-Dragon  has  also  been  evident  in  the  large 
improvement  in  target  tracking  proficiency  and  first-round  hits  in  live-fire  exercises.  The 
cost  savings  incidental  to  PGTS  are  dramatic:  one  TOW  practice  round  costs  $8.1 
thousand  while  a  PGTS  for  TOW  costs  $30.3  thousand;  one  Dragon  practice  round  costs 
$3.8  thousand  while  a  PGTS  for  Dragon  costs  $22.9  thousand.  To  sustain  gunnery  skills, 
24  practice  firings  are  required  per  month.  The  cost  of  practice  rounds  would  be  $194.4 
thousand  for  each  TOW  gunner  and  $91.2  thousand  for  each  Dragon  gunner,  compared  to 
$30  thousand  and  $23  thousand,  respectively,  for  the  entire  training  systems. 

Remoted  Target  System  (RETS) 

The  Remoted  Target  System  consists  of  stationary  and  moving  targets  that  can  be 
made  to  pop  up  as  troops  maneuver  over  a  relatively  large  exercise  area.  RETS  provides 
training  in  skills  heretofore  trainable  only  in  actual  combat  or  in  infrequent  force-on-force, 
fiee-play  exercises. 


1 0  70W:  Tube  launched,  Op)tically  Tracked,  Wire-guided  missile. 
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Combined  Arms  Staff  Trainer  (CAST) 

CAST  enables  its  users  to  expand  their  knowledge  of  supporting  arms  employment 
and  to  practice  skills  in  the  coordination  of  supporting  arms.  They  are  cost-effective 
alternatives  to  field  exercises  which,  due  to  high  cost,  are  held  infrequently. 

Tactical  Warfare  Simulation,  Evaluation,  and  Analysis  System 
(TWSEAS) 

A  heavy  demand  for  TWSEAS  exists  at  Marine  schools  and  by  combat  staffs  of  the 
Fleet  Marine  Force.  About  80  exercises— mostly  battalion-level— are  run  annually  on  four 
operational  systems.  Including  exercise  preparation  time  and  post-exercise  review,  each 
exercise  takes  one  to  two  weeks.  TWSEAS  is  a  cost-effective  alternative  to  field  exercises, 
which,  for  high-level  staffs,  are  rarely  conducted  because  the  costs  are  prohibitive; 
TWSEAS  exercises  provide  more  realism  than  command  post  exercises. 

TWSEAS  limitations:  Despite  strong  user  acceptance,  the  analog-based  TWSEAS 
will  soon  be  replaced  by  the  digital-based  Marine  Air-Ground  Task  Force  (MAGTF) 
Tactical  Warfare  Simulation  (MTWS)  that  was  developed  to  overcome  several  TWSEAS 
limitations,  which  include:  (1)  documentation  is  lacking,  making  it  difficult  to  modify  and 
maintain;  (2)  too  few  units  can  be  played;  (3)  area  coverage  is  too  limited; 

(4)  NBC  warfare,  night  operations,  and  reduced  visibility  conditions  cannot  be  played;  and 

(5)  inability  to  be  electronically  linked  to  other  Services'  models  for  joint  exercises. 

Ground  Combat  Vehicles 

Marine  Corps  Tank  Full-Crew  Interactive  Simulation  Trainer 
(MCTFIST) 

MCTFIST  is  a  portable  precision  tank  gunnery  system  that  trains  the  tank 
commander,  gunner,  driver,  and  loader  on  a  wide  range  of  gunnery  tasks  while  in  a 
stationary,  powerless  tank.  Scenery,  targets,  and  visual  effects  are  presented  through 
video  monitors  appended  to  tank  vision  blocks;  the  simulator  is  attached  to  the  actual 
vehicle. 

Training  effectiveness  and  cost  avoidance  associated  with  MCTFIST  are  illustrated 
by  the  training  experience  of  13  crews  of  a  Marine  Reserve  tank  unit  prior  to  a  live-fire 
exercise:  (1)  Two  training  days  previously  required  for  safety  training  were  saved; 


(2)  95  percent  of  the  crews  achieved  first-round  hits  on  stationary  and  moving  targets 
compared  to  a  50  to  60  percent  rate  in  pre-MCTFIST  training;  and  (3)  all  training 
requirements  were  met  with  300  fewer  than  expected  (without  MCTFIST)  main  gun  rounds 
fired  ($120,000  cost  avoidance).  MCTHST  was  acclaimed  by  a  Marine  Reserve  tank  unit 
that  received  initial  MCTFIST  training  aboard  ship  en  route  to  Southwest  Asia  during 
Desert  Shield.  They  attribute  their  combat  readiness  and  subsequent  success  in  a  tank  battle 
near  Kuwait  City  to  MCTFIST  training  prior  to  Desert  Storm.  The  success  of  MCTFIST 
has  led  to  the  adoption  of  the  appended-trainer  concept— Combat  Vehicle  Appended  Trainer 
(CVAT) — to  training  for  all  Marine  ground  combat  vehicles. 

Aircraft 

Operational  Flight  Trainers  (OFT),  Weapon  System  Trainers  (WST), 

and  Weapons  Tactics  Trainers  (WTT) 

These  flight  simulators  are  fully  integrated  into  flight  training  programs,  which 
require  a  mix  of  simulator  flying  and  actual  flying.  For  the  aircraft  listed  below,  flight 
training  programs  in  FY  1994  require  the  indicated  number  of  flight  simulator  hours  as  a 
percentage  of  total  syllabus  flight  hours  (aircraft  plus  simulator)  for  the  trainee  to  become 
combat  capable. 

Training  programs  for  four  Marine  aircraft  in  1994  require  from  13  to  37  percent 
simulator  hours  as  a  percent  of  the  total  syllabus  flight  training  hours  (aircraft  plus 
simulator)  for  the  trainee  to  become  combat  capable  (Figure  IV-4).  Helicopter  flight  is 
afforded  fewer  training  hours,  both  in  the  simulator  and  in  the  aircraft,  than  fixed-wing 
flight:  the  latter  are  advanced  aircraft. 


Aircraft 

Simulator  Hours 

Simulator  Hours 

Aircraft  +  Simulator  Hours 

AV-8B 

140.4 

37% 

F/A-18 

95.0 

28% 

CH-46 

69.0 

34% 

CH-53 

20.0 

13% 

Figure  IV-4.  Flight  Simulator  Hours  as  a  Percent  of  Total  Syllabus  Hours 
(Aircraft  Plus  Simulator)  for  Trainee  Aviator  to  Become  Combat  Capable 


Simulator  flights  must  be  performed  satisfactorily  before  follow-on  aircraft  flights 
can  be  scheduled.  Because  of  their  realism  and  their  ability  to  host  emergencies  too  risky  to 
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simulate  in  aircraft,  simulators  may  be  used  for:  (1)  up  to  50  percent  of  minimum  flying 
hour  requirements;  (2)  flights  to  evaluate  operational  procedures;  (3)  instrument  and 
NATOPS”  check  flights;  and  (4)  50  percent  of  minimum  flying  requirements  for 
instrument  rating. 

The  Marine  Corps  has  27  flight  simulators  that  cost  $394  million  (then-year 
dollars).  In  the  case  of  two  aircraft,  the  investment  in  simulators  is  2  to  5  percent  of  the 
investment  in  aircraft  (Figure  IV-5).  Both  aircraft  and  simulator  hours  in  flight  training 
programs  have  increased  over  the  last  ten  years,  with  a  larger  increase  for  simulator  hours 
(Figure  IV-6).  The  proportion  of  simulator  hours  in  this  total  has  also  increased,  except  for 
the  CH-53E. 


Number  of 
Aircraft 

Total 

Cost 

Number  of 
Simulators 

Total 

Cost 

Simulator  $ 

Aircraft  $ 

283  AV-8 

$6,606  million 

6 

$117  million 

2% 

116AH-1W 

$1,104  million 

3 

$  56  million 

5% 

Figure  IV-5.  Relative  Investments  in  Aircraft  and  Simulators,  for  the  AV-8  and 

AH-1W  aircraft 


Aircraft  Hours 

Simulator  Hours 

Simulati 

Aircral 

Dr  Hour 

t  Hours 

s 

Aircraft 

1983 

1986 

1991 

1994 

1983 

1986 

1991 

1994 

1983 

1986 

1991 

1994 

F/A-18 

216.3 

243.1 

75.3 

95.0 

.35 

.39 

AV-8 

164.8“ 

188.7 

234.7‘> 

12 

74.5 

140.4 

.07 

.39 

.60 

CH-46 

153.5 

121.0 

129.5 

131.5 

30.0 

{36.0)= 

16.0 

(20.5) 

69.0 

(7.5) 

.43 

.33 

.28 

.58 

CH-53E 

147.0 

145.0 

136.5 

128.5 

B 

15.0 

(7.5) 

18.0 

(11.0) 

20.0 

(0) 

■ 

.16 

.21 

.16 

a  AV-8A. 

^  Radar-equipped  AV-8B. 

^  Parentheses  indicate  simulator  flight  may  be  used  if  available. 

^  Simulator  not  available. 

Figure  IV-6.  Flight  Training  Programs  in  the  Marine  Corps  for  Basic  Pilots 
•  To  Become  Fully  Combat  Qualified 
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Naval  Air  Training  and  Operating  Procedures  Standardization. 
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Simulator  flying  is  mandatory  in  fixed-wing  training  programs.  In  helicopter 
training  programs,  a  large  part  of  simulator  flying  has  been  optional,  i.e.,  some  training 
missions  can  be  flown  in  simulators,  if  they  are  available,  or  in  aircraft. 

Two  shortcomings  have  made  the  use  of  simulators  less  vital  to  helicopter  flight 
training  than  to  fixed-wing  flight  training: 

1 .  Low-fidelity  video  poorly  represents  terrain,  vegetation,  and  other  external 
objects  in  the  low-level  flight  regime,  where  helicopters  operate  most  of  the 
time. 

2 .  Simulators  poorly  imitate  the  stability  and  control  of  helicopters;  helicopters  are 
easier  to  fly  then  their  simulators. 

Maintenance 

Universal  Maintenance  Training  Systems  (UMTS) 

UMTS  is  a  networked  panel  trainer  with  software  and  video  that  allow  an  instructor 
to  train  students  at  21  individual  stations  at  one  time.  UMTS,  which  makes  maximum  use 
of  commercial  off-the-shelf  hardware  and  software,  provides  training  for  operation  and 
maintenance  of  complex  equipment  without  the  need  for  actual  equipment.  The  generic 
nature  of  the  system  enables  it  to  be  adapted  to  new  courses  at  a  fraction  of  the  cost  of 
developing  a  new  trainer.  The  interactive  nature  of  UMTS  provides  an  environment  that 
helps  turn  out  students  with  consistent  knowledge  and  capabilities. 

Findings:  Marine  Corps 
Successes 

Strong  user  acceptance;  heavy  usage. 

Simulations  are  vital  parts  of  ground  and  aviation  training  programs. 

Portability  is  an  important  characteristic  of  some  training  devices,  that  can  be 
used  to  exercise  skills  while  deployed  and/or  embarked  aboard  ship. 

Pervasive  opinion  of  Marine  Corps  officers  (Lt  Colonels  and  Colonels)  is  that 
Marines  trained  with  simulators  today  are  much  better  trained  than  Marines 
who  trained  in  the  past  without  them. 

Problems 

Cost  of  converting  present  closed-loop  simulations  to  DIS  compatibility. 
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A  review  of  Air  Force  experience  with  simulation  was  facilitated  by  the  fact  that  this 
subjfcct  was  evaluated  recently  at  the  highest  level  and  reported  in  "Four-Star  Flight 
Simulator  Review"  (Department  of  the  Air  Force,  10  May  1993).  This  is,  obviously,  an 
authoritative  document  and  much  of  the  information  that  follows  is  derived  from  that 
report.  The  purpose  of  that  review  was: 

Examine  Air  Force  flight  simulator  policies  and  programs  to  identify  cost- 
effective  approaches  to  flight  simulation  and  obtain  direction  on  future  flight 
training  simulator  investment  strategies. 

Guidance  on  the  use  of  simulators  by  the  Air  Force  was  provided  by  an  earlier 
review  (Department  of  the  Air  Force,  1984)  and  directive  (1985),  as  follows: 

Simulators  complement  flying  time  Simulation  Broad  Area 

Review,  1984 

Concentrate  .simulator  training  on  tasks  that  cannot  be 
trained  effectively  in  the  aircraft.  Because  of  high 
cost,  limit  sophistication  of  flight  simulators 

Adapt  commercial  training  where  appropriate 

Develop  system  training  plans 

Use  contractor  logistics  support  to  free  up  main-  Program  Management 
tp.nanri*  personnel  to  support  direct  combat  needs  Directive  5220, 1985 

Phase  out  aircrew/missile  training  device  career  field 

The  Air  Force  is  committed  to  a  simulator  development  program  that  will  cost  about 
$2.2  billion,  including  RDT&E  and  procurement.  About  equal  amounts  (about  $0.7  billion 
each)  will  be  spent  for  training  systems  for  air  crew  and  weapon  systems  (Figure  rV-7). 


^2  We  acknowledge  the  assistance  of  the  following  individuals  who  were  most  helpful  in  obtaining 
information  on  simulation  and  training  in  the  Air  Force: 

James  D.  Basinger.  James  J.  O'Connell  and  James  Brown,  Air  Force  Training  System  Program  Office; 
Patrick  Bowden  and  Robert  Denton,  Air  Force  Education  and  Training  Command;  and  Martha  Weller, 
Institute  of  Aviation,  University  of  Illinois. 
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Training  Systems 

RDT&E  ($  Millions) 

Procurement  ($  Millions) 

Aircrew 

$310.5 

$409.0 

Maintenance 

290.6 

329.9 

Electronic  Warfare 

9.9 

31.8 

Weapon  system 

217.7 

520.4 

Development  activities 

94.9 

Total 

$923.6 

$1,291.1 

Figure  IV-7.  Air  Force  Simuiator  Deveiopment  Program 
(Source:  Four-Star  Flight  Simulator  Ravlaw,  1993) 

Deficiencies  of  current  and  still-to-be  delivered  simulators,  as  reported  by  the  major 
commands,  are  shown  in  Figure  IV-8. 


Command 

Deficiency 

Air  Training  Command 

Joint  Primary  Aircrew  Training  System/ 
Ground-Based  Training  System 

None  (system  available  FY  98) 

Electronic  Warfare  T4/T5  PTT 

No  devices  for  training 

(1)  suppression  of  enemy  air  defense  (SEAD) 
principles 

(2)  integrated  electronic  combat 

Air  Mobiiity  Command 

Joint  Primary  Aircrew  Training  System/ 
Ground-Based  Training  System 

Inadequate  visual  system 

Inadequate  receiverAanker  interaction  models 

Electronic  Warfare  T4/T5  PTT 

Insufficient  WSTs  to  meet  training  demands 

C-5WSTARPTT 

Inadequate  visual  system 

Poor  receiverAanker  interaction  model 

Insufficient  WSTs  to  meet  training  demands 

C-141  WST.ARPTT 

Inadequate  visual  system 

Poor  receiverAanker  interaction  model 

Navigator  unable  to  perform  air  drop  duties  in  WST 

C-130E/H  WST 

Aero  model  differs  from  a/c  at  low  altitudes  and  on  the 
ground 

C-130H-2AWST 

No  WST  configured  for  this  a/c 

C-130H-3 

Available  simulators  inadequate  for  systems  peculiar  to 
this  a/c 

C-17 

(Awaiting  flight  test  aero  data) 

(continued) 


Figure  iV-8.  Evaiuation  of  Air  Force  Simuiators 


IV-22 


Command 

Deficiency 

Air  Mobility  Command  (cont'd) 

KC-10 

Computer  cannot  support  further  modifications  to  the 
system 

KC-135 

inadequate  visual  system 

Excessive  computational  lags 

Performance  differs  from  a/c 

No  motion 

Special  Operations  Command 

SOF  Aviation  Training  System 

Unanticipated  cost  growth 

Required  funding  unavailable 

Training  materials  not  available 

Combat  Air  Command 

F-22 

None  (awaiting  delivery) 

F-15/F-16 

Primarily  a  procedures  trainer 

Does  not  provide  required  training 

Expensive  to  operate  and  maintain 

A/OA-10 

Requited  simulators  not  available  since  retirement  of 
low  fidelity  OFTs 

F-15A/C 

No  visual  cues 

F-16A/B 

Lacks  visual  cues  for  practicing  out-of-control 
recoveries 

F-15E 

Lacks  visual  cues  (full  dome  was  canceled) 

B-1 

None  identified 

B-52  WST/OSMT 

Cannot  train  conventional  tactics  and  weapons  delivery 

B-2 

None  identified  (awaiting  delivery) 

JSTARS 

Training  uses  KC-13J 

JSTARS-modified  WST  does  not  exist 

Figure  IV-8.  Evaluation  of  Air  Force  Simulators  (cont'd) 


As  reported  in  the  Four-Star  review,  two  major  commands  take  different  positions 
on  the  use  of  flight  simulators  for  air  crew  training: 

Air  Mobility  Command 

Use  flight  simulators  for  aircrew  training. 

Reduce  expensive  flying  hours,  by  up  to  50  percent. 

Increase  use  of  simulators  as  years  of  pilot  experience  increases. 


IV-23 


Use  simulators  for  maintenance  training. 

Upgrade  training  devices  first  that  have  largest  return  on  investment  (C-5, 
C-141). 

Air  Combat  Command 

Use  aircrew  training  devices  to  complement  rather  than  supplement  flying 
hours. 

Widely  dispersed  small  units  dictate  low-cost,  high-fidelity  simulators  for 
training  pilots. 

Replace  flight  simulators  with  low-cost  unit  training  devices. 

Use  simulators  to  train  maintenance  and  part-task  trainers  for  electronic 
warfare,  IR  target  recognition,  digital  landmass. 

An  emerging  concept  in  the  Air  Force  is  that,  where  feasible,  current  generation 
flight  trainers  should  be  replaced  with  low-cost  training  devices.  Because  of  their  high 
cost,  only  an  insufficient  number  of  flight  trainers  can  be  procured  and  some  pilots  have  to 
travel  to  other  bases  to  use  them.  Given  the  advances  in  computers  and  the  reduction  in 
their  cost,  the  Air  Force  plans  to  develop  multi-task  trainers  with  a  single  visual  channel 
that  can  be  used  for  various  training  purposes,  e.g.,  emergency  procedures,  instrument 
flight,  approach,  air-to-air  tactics,  IR,  and  use  of  digital  land  mass  data.  It  is  estimated 
that  unit-level  training  devices  to  support  F-15  and  F-16  aircraft  cost  about  $0.7  to 
$1.25  million  each.  The  Air  Force  plans  to  procure  97  such  devices  and  place  one  or  more 
at  each  flight  base.  A  comparison  of  the  cost  of  ownership  of  current  operational  flight 
trainers  and  the  proposed  unit  training  devices,  as  shown  in  Figure  IV-9,  suggests  that,  due 
to  savings,  the  total  cost  of  97  devices  ($67.9  to  $121.3  million)  could  be  amortized  in 
0.9  to  1 .7  years. 

The  Four-Star  Flight  Simulator  Review  concludes  that: 

Most  weapons  systems  do  not  have  system  training  plans. 

There  are  data  base  deficiencies  in  the  aero  model  in  14  of  43  simulator 
models. 

Standardized  data  bases  are  necessary  for  realistic  training. 

Simulator  technology  is  available  which  can  provide  high-quality  simulation  at 
low  unit  cost. 


Lack  of  currency  between  simulators  and  aircraft  is  a  major  concern  of  all 
MAJCOMS. 


Summary:  Air  Force  Simulations 

1 .  The  Air  Mobility  Command  is  willing  to  use  flight  simulators  for  aircrew 
training  and  replace  training  flying  hours  by  up  to  50  percent.  It  will  upgrade  training 
devices  starting  with  those  that  project  the  largest  return  on  investment. 

2 .  The  Air  Combat  Command  uses  flight  simulators  to  complement  rather  than  to 
supplement  flying  hours.  It  plans  to  replace,  rather  than  upgrade,  current  flight  simulators 
with  97  low  cost  unit  training  devices  and  locate  them  at  flying  bases. 

3 .  Most  deficiencies  in  existing  simulators  are  the  result  of  decisions  not  to 
upgrade  existing  equipment  or  not  to  provide  certain  capabilities,  e.g.,  improved  visuals, 
motion  bases,  more  units,  or  more  capability  (e.g.,  IR,  EW).  Current  technology  could 
meet  most  of  these  deficiencies,  if  so  desired.  The  issue  is  policy  and  budget  rather  than 
technology. 


FINDINGS 


Successes 

High  user  acceptance 

Training  program  well  developed 

Use  of  simulation  integrated  into  training  program 

Acceptable  fidelity  (e.g.,  target  signals,  maneuverability,  visual  display,  simulation 
of  g  forces)  provides  a  good  combat  environment 

Trains  emergency  procedures 

Provides  feedback  for  training 

Provides  cost  savings 

No  acceptable  alternative  for  training 

Problems 

Inadequate  or  no  training  program 

Various  simulators  do  not  provide  conditions  needed  for  training,  e.g., 
jamming 
ground  threats 
multiple  aircraft  targets 
submarines 

encrypted  links  to  other  platforms 
semi-automated  forces 
integrated  electronic  combat 
air  drop  duties  for  navigator 
compatibility  with  DIS 

NBC  warfare,  night  operations,  reduced  visibility,  compatibility  with  DIS 
Inadequate  simulation 
visual  system 
aerodynamic  model 

receiverAanker  interaction 
low  altitude  effects 


on-the-ground  movements 
performance  recording  and  feedback 
mission  rehearsal  c^ability 
approaching  targets  only;  need  lateral  moving  targets 
mismatrh  between  ammo  flash  and  electro-optical  sight  (no  blooming) 
documentation  needed  for  maintenance  and  modification 
area  coverage 

Too  expensive  to  operate  and  maintain 

Not  portable  (for  carrier  or  deployed  use) 

Insufficient  units  to  meet  training  demands 

Excessive  computational  lags 

Computer  cannot  support  further  modifications  to  system 

Service  Experience  with  Simulations 

1 .  Most  deficiencies  in  existing  simulators  are  the  result  of  decisions  not  to 
upgrade  existing  equipment  or  not  to  provide  certain  capabilities,  e.g.,  improved  visuals, 
motion  bases,  more  units,  more  capability  (e.g.,  IR,  EW).  Current  technology  could  meet 
most  of  these  deficiencies,  if  so  desired.  The  issue  is  policy  and  budget  rather  than 
technology. 

2.  Successful  simulators  have  adequate  training  plans,  fidelity  adequate  to  the 
training  plan,  provide  feedback  for  training,  provide  training  not  possible  any  other  way 
(e.g.,  emergency  procedures),  and  save  costs. 

3 .  Simulators  reported  to  have  problems  tend  to  lack  a  training  plan,  capabilities 
that  appear  desirable  now  but  which  were  not  provided  originally  (e.g.,  jamming,  multiple 
vehicles,  adequate  documentation,  adequate  performance  measurement,  portability)  and  are 
expensive  to  operate  and  maintain. 

4 .  The  following  trends  are  discernible: 

Air  Mobility  Command  -  Use  simulators  to  reduce  flying  hours  up  to 

50  percent 

Air  Combat  Command  -  Complement  rather  than  supplement  flying  hours 

Replace  flight  simulators  with  many  low-cost 
unit  training  devices 


Use  part-task  trainers  for  electronic  warfare,  IR 
target  recognition,  digital  landmass 

-  Use  simulators  to  reduce  flying  hours  up  to 
50  percent 

-  Use  simulators  to  reduce  vehicle  miles  per  year 
(SIMNET/CCTT) 

-  Modifying  existing  simulations  and  simulators  so 
that  they  can  interact  with  other  simulators  and 
simulations  of  their  own  and  other  Services 

Developing  distributed  interactive  simulation 
systems  so  that  all  types  of  simulations 
(simulators,  live  exercises  and  combat  models) 
can  participate  in  large-scale  Service  and  joint 
exercises 

5 .  If  simulators  are  as  effective  as  actual  equipment  for  training  personnel  on 
certain  specific  tasks,  as  the  data  show,  this  could  lead  the  Services  to  a  reduction  of 
OPTEMPO,  i.e.,  because  of  cost  savings,  and  an  increased  utilization  of  simulators.  The 
Services  differ  in  their  response  to  this  opportunity:  The  Air  Mobility  Command,  Marine 
Corps,  and  the  Army  are  increasing  the  use  of  simulators  (flight  and  SIMNET/CCTT)  and 
decreasing  OPTEMPO  (flying  hours  and  vehicle  miles);  the  Air  Combat  Command  and  the 
Navy  use  flight  simulators  to  enhance  but  not  reduce  flying  hours  (except  by  a  small 
amount).  Little  emphasis  is  given  to  the  development  of  simulations  for  joint  Service 
combat  exercises,  except  for  the  ARPA  and  DMSO  programs,  such  as,  the  Strategic 
Theater  of  War  (STOW)  and  the  Multi-Service  Distributed  Training  Testbed  (MDT2):  the 
Services  do  participate  in  these  development  programs. 


Marine  Corps 
Army 

All  Services 


V.  DISTRIBUTED  INTERACTIVE  SIMULATION 


Distributed  Interactive  Simulation  (DIS)  is  a  current  advance  in  technology  that  has 
great  potential  for  improving  training  at  unit.  Service,  and  joint  levels.  It  builds  on  the  now 
familiar  concept  and  technology  demonstrated  in  the  Simulator  Networking  (SIMNET) 
program  that  DARPA  and  the  Army  started  in  1983.  It  depends  heavily  on  standard 
protocols  needed  to  support  real-time  communications  between  many  participants  (in  the 
thousands)  over  intelligent  networks;  this  type  of  capability  is  also  called  Advanced 
Distributed  Simulation  (ADS).  Put  simply,  it  creates  an  environment  in  which  units  of  all 
sizes  (e.g..  Army  and  Marine  battalions,  brigades,  or  divisions.  Air  Force  squadrons,  and 
Naval  task  forces)  separately  or  in  combination,  can  engage  in  two-sided,  real-time  combat 
exercises,  very  much  as  if  they  were  involved  in  a  conventional  large-scale  field  exercise  on 
a  well-instrumented  range.  The  participants  in  DIS  exercises,  regardless  of  their  actual 
locations,  may  be  in  simulators  or  in  actual  equipment,  in  simulated  or  in  actual  command 
and  control  centers,  and,  with  some  acting  as  an  enemy,  engage  in  a  large-scale  combat 
exercise  on  a  common  geographical  location  (which  might  be  that  of  an  assumed  enemy). 
The  environment  is  a  synthetic  one  for  personnel  in  simulators  or  for  those  in  command 
posts  who  interact  with  computer-based  combat  models  and  a  real  environment  for  those  in 
the  field  exercise  who  may  not  know  that  some  participants  are  in  simulators.  All 
participants  operate  as  if  they  were  located  in  the  same  geographical  area;  those  in  the  live 
exercise  are  actually  there.  This  type  of  capability,  although  considered  here  only  for 
purposes  of  training,  has  applications  for  mission  rehearsal,  test  and  evaluation  of  new 
equipment  and  weapons,  and  development  and  test  of  doctrine  and  tactics  in  a  way  that 
does  not  expose  such  activities  to  external  observation. 

The  Defense  Science  Board  Task  Force  on  Simulation,  Readiness  and  Prototyping 
strongly  supported  the  development  of  distributed  interactive  simulation  and  recommended 
twelve  demonstrations  intended  to  improve  training  readiness,  create  a  joint  environment 
and  provide  support  to  acquisition  (Defense  Science  Board,  1993).  Six  of  the  12 
demonstrations  are  directed  to  improve  training  readiness: 

Joint  Task  Force  campaign  planning  and  training 

Interactive  exercises  at  home  stations 


Integrated  National  Guard  Brigade  training 
CINC  wargaming  networking 
Network  training  and  test  ranges 
Realistic  electronic  combat  test  and  training. 

The  viability  and  wide-scope  capability  of  Distributed  Interactive  Simulation  is 
attested  to  by  the  fact  that  at  least  26  demonstrations,  exercises,  or  tests  of  this  technology 
have  been  held  since  1987  or  are  planned  for  the  near  future;  some  have  included  forces  in 
Europe  and  Korea  (Figure  V-1). 

Current  DIS  development  programs  that  relate  specifically  to  training  include  the 
foUowing: 

Estimated  Program  Budget^ 


Close  Combat  Tactical  Trainer  (CCTT) 

$  2.1 

billion 

Aviation  Combined  Arms  Tactical  Trainer  (AVCATT) 

1.5 

billion 

Synthetic  Theater  of  War  (STOW) 

350 

million 

National  Guard  Training 

60 

million 

Multi-Service  Distributed  Training  Testbed  (MDT2) 

11 

million 

Simulation  Technology  Development 

98 

million 

The  AVCATT  is  a  prospective  program.  This  list  does  not  include  many  programs  of  the 
Services,  DMSO,  and  ARPA  that  are  developing  the  technology  needed  to  support  DIS 
(e.g.,  networks,  virtual  reality),  but  that  are  not  specifically  aimed  at  training.  Some  of 
these  are  considered  below. 

Training  exercises  conducted  with  DIS  are  likely  to  be  cost-effective,  compared  to 
live  field  exercises,  although  we  have  not  seen  an  analysis  of  this  issue.  According  to 
Noble  and  Johnson  (1991),  the  total  life  cycle  cost  of  546  CCTT  simulators  for  company 
and  team  training  is  estimated  to  be  $1.19  billion  (FY  92  constant  dollars).  This  cost  can 
be  "paid  back"  over  15  years  by  a  reduction  in  OPTEMPO  of  61  miles  per  tank  per  year 
(7.6  percent)  from  800  miles  for  the  Active  Component  and  45  miles  per  tank  per  year 
(15.6  percent)  from  288  miles  for  the  Reserve  Component  (Noble  and  Johnson,  1991). 
The  total  life  cycle  cost  of  844  CCTT  simulators  for  Battalion  Task  Force  training  for  the 
Active  Component  and  114  simulators  for  the  Reserve  Component  is  estimated  to  be 
$2.1  billion  (FY  92  constant  dollars).  The  payback  over  15  years  would  require  a 


*  Source;  Simulation  Initiative  Reply,  DoD  PA&E  Memorandum,  Dec.  21, 1993  and  other  sources. 


reduction  in  OPTEMPO  of  114  miles  per  tank  per  year  (14.3  percent)  for  the  Active 
Component  and  45  miles  per  tank  per  year  (15.6  percent)  for  the  Reserve  Component. 


EXERCiSE 

AGENCY 

DATE 

CATC-87 

DARPA/ARMY/ARI 

Mar-Jun  87 

FAADS/ADATS  FDT&E 

TEXCOM/USADAS&C 

Mar-Apr  88  and  Sep  89 

USAFE  JAAT 

TAC/USAFE 

Mar  89 

WARE-03/90 

ARPA/Army 

Mar  90 

BIT 

ARPA/Navy 

Jun  90 

73  Easting 

ARPA 

91 

13th  l/ITSEC 

Army 

Nov  91 

CTAS 

Navy/UK 

Nov  91 

SASC  Testimony 

DDR&E 

May  92 

Ulchi  Focus  Lens 

Joint-Korean 

Aug  92 

REFORGER  ‘92 

ARP/\/Army 

Sep-Oct  92 

War  Breaker  Zealous  Pursuit 

ARPA 

Oct-Dec  92 

Louisiana  Maneuvers  IPR 

Army 

Oct  92 

Maritime  UAV 

ARPA/JPO-UV 

Sep-Dec  92 

14th  l/ITSEC 

ADPA/Air  Force 

Nov  92 

Missile  Defense 

DDR&E 

Mar  93 

Maritime  Demo 

ARPA 

Sep  93 

15th  l/ITSEC 

ARPA/Navy/OSTP 

Nov  93 

STOW  Engr  Demo 

ARPA 

Dec  93 

INCOMSS-94 

DDR&E/Army 

Feb  94 

SAFAGANZA 

ARPA 

Feb-Jul  94 

MDT2 

ARPA/STRICOM 

Mar-May  94 

AUSA 

DMSO/Army 

May  94 

Capital  Hill  Support 

DDR&E 

Jun  94 

STOW-E  (REFORGER) 

ARPA/Army 

Oct-Nov  94 

leth  l/ITSEC 

AD  P  A/Army 

Nov  94 

Figure  V-1.  Tests  and  Demonstrations  of  Distributed  interactive  Simuiation, 

1987-1994 


Regardless  of  the  reliance  placed  on  the  use  of  CCTT,  or  more  generally  DIS,  for 
training,  it  is  useful  to  remember  that  some  field  training  would  stUl  be  needed,  not  only  to 
build  experience  with  and  confidence  in  the  use  of  the  actual  equipment  to  be  used  in 
combat  but  to  provide  training  in  tasks  not  intended  for  and  not  provided  for  in  the  design 
of  the  CCTT  system.  It  is  estimated  that  the  CCTT  can  provide  training  for  about 
60  percent  of  the  armor  tasks  specified  in  the  Army  Training  Evaluation  Program  (Noble 
and  Johnson,  1991). 

CCTT  is,  of  course,  an  improved  version  of  SIMNET;  several  tests  conducted  in 
SIMNET  suggest  that  CCTT  will  be  an  effective  training  device  (See  Figure  V-2).  When 
measured  by  the  scores  obtained  on  55  tasks  specified  by  ARTEP,  platoons  trained  in 
SIMNET  performed  as  well  in  three  field  exercises  as  those  who  received  conventional 
field  training  (Gound  and  Schwab,  1988).  A  further  analysis  of  the  data  in  this  report 
shows  that  SIMNET  training  on  tasks  that  were  judged  fully  trainable  by  SIMNET 
significantly  improved  the  post-training  performance.  For  a  given  subset  of  tasks  that  are 
fully  represented  in  SIMNET,  and  within  a  given  amount  of  time,  SIMNET  training  is 
more  effective  than  additional  field  exercises  (Angier,  Alluisi  and  Horowitz,  1992,  p.  40 
and  Appendix  B).  Platoons  with  more  battle  runs  on  SIMNET  produced  higher  scores  in 
the  competition  for  the  Canadian  Armor  Trophy  (Kraemer  and  Bessemer,  1987).  SIMNET 
training  on  a  field  scenario  improved  field  performance  (TEXCOM,  1990);  SIMNET 
training  improved  field  performance  ratings  in  the  Armor  Officer  Basic  Course  over 
platoons  that  received  conventional  training  (Bessemer,  1991).  No  known  instance  of 
training  in  SIMNET  produced  negative  results. 

Nevertheless,  issue  may  be  taken  with  these  evaluations  on  the  following  grounds: 

•  Small  sample  size:  Same  performance  of  SIMNET  and  non-SIMNET  trained 
groups,  i.e.,  no  significant  difference  between  the  groups,  may  be  due  to 
small  sample  size  rather  than  to  SIMNET  training. 

•  No  baseline:  Control  group  (conventional  training)  needed  as  a  baseline  to 
estimate  benefit  of  SIMNET  training  (except  Ft.  Hood  test). 

•  Inadequate  test  design:  Improved  performance  "due  to  SIMNET"  may  be  due 
to  additional  field  training  (TEXCOM,  Ft  Knox  tests),  increased  experience  of 
instructors,  changes  in  rating  scale,  better  student  quality  (Ft  Knox  test). 


SIMNET  training  for 
Canadian  Armor 
Trophy  Competition 


SIMNET  training  vs. 
conventional  field 
training  at  Ft.  Hood 


Additional  analysis  of 
above  data:  10  tasks 
common  to  SIMNET 
and  8  STXs  _ 


Field  exercise  plus 
SIMNET  training  on 
same  scenario: 
movement-to-contact 


Performance  ratings  in 
Armor  Officer  Basic 
Course,  Ft.  Knox, 
1987-1989 


SIMNET  capability  to 
train  ARTEP  MTP 
tasks  (analysis,  not 
test) 


CCTT  capability  to 
train  ARTEP  MTP 
tasks  (analysis,  not 
test)  _ 


British  combat  trials  in 
,  AGPT _ _ 

Operational  tests  of 
anti-helicopter  smart 
mines  vs.  helicopters 
in  SIMNET  (not  tests 
of  training) 


ARTEP  Army  Training  Evaluation  Program 
MTP  Mission  Training  Program 

AGPT  German  SIMNCT 

STX  Situational  Training  Exercise 

CCTT  Close  Combat  T actical  T rainer 


No.  of  Cases 

Finding 

Source 

9  platoons 

More  SIMNET  battle  runs 
produced  higher  scores  in 
CAT  competition  (r=0.53) 

Kraemer  and 
Bessemer,  1987 

8  platoons 

No  difference  in  scores  on 
55  tasks  in  3  instrumented 
field  exercises 

Gound  and 
Schwab,  1988 

8  platoons 

SIMNET  group  performed 
better 

Brown,  Piskel, 
and  Southard, 
1988 

9  armor  and  9 
mechanized  infantry 
platoons 

All  groups  showed 
improved  performance 

TEXCOM,  1990 

714  platoons, 
conventional  training 

39  platoons,  SIMNET 
training 

SIMNET  Improved  field 
performance  ratings  by  25 
percent  and  saved  20 
percent  of  time  in  course 

Bessemer,  1991 

SIMNET  can  provide 
training  on  about  35%  of 
ARTEP  tasks 


Burnside,  1990 


CCTT  can  provide  training  Noble  and 
on  about  60%  of  ARTEP  Johnson,  1991 
tasks 


AGPT  reduces  navigation  Kelly,  1994 
errors  by  60  percent 


46  offensive  and  40 
defensive  trials 


Helicopter  commanders 
improved  their  ability  to 
avoid  mines  by  35  percent 
and  mine  layers  improved 
deployment  by  37  percent 
(exchange  ratio 
improvements  from  first 
half  to  second  half  of  trials) 


Schwartz  and 
DeRiggi  ,1994 


Figure  V-2.  Summary  of  Studies  Showing  the  Effectiveness  of  SIMNET  for 

Training 

Boldovici  and  Bessemer  (1993,  draft)  provide  an  excellent  discussion  of  the 
problems  encountered  in  trying  to  evaluate  SIMNET.  Of  great  importance,  obviously,  is 
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how  to  evaluate  the  utility  of  CCTT  for  training  and  of  other  forthcoming  DIS  systems  like 
STOW,  MDT2  and  the  National  Guard  system.  These  DIS  systems,  almost  by  definition, 
operate  with  large  numbers  of  people;  the  staff  needed  to  develop  and  run  a  test  and  collect 
and  evaluate  the  available  data  is  not  negligible  in  size.  Ultimately,  performance  of  the 
group  trained  in  CCTT  must  also  be  measured  in  field  trials,  so  that  we  can  compare  the 
field  performance  of  troops  trained  conventionally  with  those  trained  using  CCTT.  We 
believe  that  insufficient  attention  has  been  given  to  develop  test  plans  needed  to  evaluate  the 
effectiveness  and  cost  of  DIS  systems  now  being  developed. 

MULTI-SERVICE  DISTRIBUTED  TRAINING  TESTBED  (MDT2) 

A  significant  program  concerned  directly  with  the  utility  of  distributed  interactive 
simulation  for  joint  training  is  the  Multi- Service  Distributed  Training  Testbed  (MDT2).  The 
purpose  of  this  program  is  to: 

1 .  Develop  a  realistic  synthetic  environment  primarily  for  training,  but  able  to 
support  mission  rehearsal,  execution,  and  feedback;  the  initial  application  is 
Close  Air  Support. 

2.  Develop  and  test  training  scenarios,  assessment,  and  feedback  for  Close  Air 
Support. 

This  is  a  multi- Service,  collaborative  effort  with  participation  by  all  four  Services; 

Air  Force  Armstrong  Laboratory,  Mesa,  AZ 

Army  Research  Institute,  Alexandria,  VA 

Naval  Training  Systems  Center,  Orlando,  FL 

Marine  Corps  Combat  Development  Command,  Quantico,  VA 

SIMNET,  Ft.  Knox,  KY 

Tactical  Operations  Center,  Mounted  Warfare  Testbed,  Ft.  Knox,  KY 
Naval  Research  and  Development  Activity,  San  Diego,  CA 
Patuxent  Naval  Air  Test  Center,  MD 

The  budget  for  this  program  is  a  total  of  $10.5  million  for  FY  1993  and  FY  1994 
(Figure  V-3).  Each  service  contributes  approximately  equal  amounts;  about  40  percent  is 
contributed  by  the  Defense  Modeling  and  Simulation  Office  (DMSO).  When  DMSO 
support  ends  after  two  years,  it  is  anticipated  that  the  Services  will  continue  to  support 
MDT2.  Then,  work  on  Close  Air  Support  would  be  followed  by  training  on  such  missions 
as  Joint  Air  Defense,  Combined  Amphibious  and  Land  Assault,  and  Joint  Fire  Support. 


FY93 

FY94 

SERVICES 

DMSO 

SERVICES 

DMSO 

ARMY 

$  1380  K 

$  1430  K 

$  1010  K 

$425  K 

NAVY 

1100 

620 

810 

300 

AIR  FORCE 

1200 

840 

1000 

410 

TOTAL 

$3680  K 

$  2890  K 

$  2820  K 

$  1135  K 

Figure  V-3.  Budget  for  the  Multi-Service  Distributed  Training  Testbed  (MDT2) 

for  FY  1993  and  FY  1994 


The  MDT2  program  is  correctly  called  a  Testbed  and  it  may  in  time  develop  into 
both  a  training  and  training  evaluation  program.  Over  the  initial  two  years,  about 
$6.5  million  (60  percent)  will  go  for  the  development  and  test  of  the  hardware  and  software 
needed  to  operate  the  Testbed.  About  $4.0  million  (40  percent)  will  be  given  to  the 
development  of  scenarios  and  the  conduct  of  tests  of  Close  Air  Support  (CAS)  needed  to 
collect  and  evaluate  preliminary  cost  and  effectiveness  data  on  the  utility  of  this  prototype 
for  joint  training.  After  the  Testbed  has  been  developed,  it  is  estimated  that  65  percent  of 
all  expenditures  will  be  for  tests  of  training  with  the  residual  (35  percent)  needed  to  sustain 
operation  and  upgrade  of  the  Testbed.  Future  programs  supported  by  the  Services  are 
estimated  to  be  of  the  order  of  $2  to  $3  million  per  year.  Due  to  limitations  of  time  and 
funds,  the  program  to  test  the  capability  of  the  Testbed  for  training  the  CAS  mission  will  be 
based  on  performance  measures  within  the  Testbed  before  and  after  training  during  a  period 
of  five  days;  there  will  be  about  30  people  in  30-60  trials.  This  test  design  does  not  permit 
an  estimate  of  how  much  training  in  the  Testbed  carries  over  to  performance  on  an 
instrumented  range  or  how  well  those  trained  in  the  simulator  perform  CAS  on  a  range  in 
comparison  with  those  trained  only  on  a  range.  Examination  of  training  strategies,  for  the 
purpose  of  finding  which  methods  of  feedback  of  performance  data  best  serve  to  improve 
performance,  is  also  left  for  future  work. 

Some  Service  efforts  that,  though  not  performed  as  inter- Service  projects,  are 
focused  on  collective  training  and,  by  extension,  can  contribute  to  joint  training,  are 
considered  here.  The  Air  Force  has  linked  two  F-15C  cockpits,  an  air  weapons  controller 
station,  two  enemy  cockpits  and  two  dome  display  systems  in  a  distributed  interactive 
simulation  system,  based  on  SIMNET-like  communication  technology  (Bell  and  Crane, 
1993).  The  system,  called  MultiRAD,  for  Multiship  Research  and  Development,  is  located 
at  the  Aircrew  Training  Research  Division  of  the  Armstrong  Laboratory,  Mesa,  Arizona; 


MultiRAD  also  serves  as  the  Air  Force  component  of  the  Multiservice  Distributed  Training 
Testbed  (MDT2).  Four  one-week  exercises,  with  F-15  pilots  and  13  air  weapons 
controllers  on  267  multiship  missions,  found  the  cockpits  wholly  acceptable  for  air  combat 
training;  the  cockpits  lacked  rudder  panels  and  had  "glass"  touch  panels  (computer¬ 
generated  instrument  displays).  Tasks  that  were  rated  higher  ("better")  in  MultiRAD 
training  were  employment  of  electronic  countermeasures,  employment  of  chaff  and  flares, 
defense  against  surface-to-air  missiles,  work  with  an  air  weapons  controller,  and 
engagement  against  four  or  more  enemy  aircraft.  Tasks  rated  lower  in  MultiRAD  than  in 
current  unit  flight  training  were;  tactical  formation,  visual  lookout,  mutual  support,  and 
visual  low  altitude  flight.  These  are  tasks  that  require  visual  imagery  (beyond  current 
simulator  capability)  and  precise  handling  qualities  of  the  aircraft  The  lessons  learned  are 
based  on  pilots'  comments  and  evaluations,  not  on  performance  measures  in  aircraft  after 
MultiRAD  training. 

The  Army  Research  Institute  has  developed  a  Unit  Performance  Assessment 
System  (UPAS)  that  collects  network  data  from  SIMNET  exercises  and  provides  a  variety 
of  summary  data  for  use  in  after-action  reviews  (Meliza,  Bessemer,  and  Tan,  1994).  These 
data  include  map  displays  with  terrain  features  and  contour,  the  overall  battle  space  at  any 
instant,  line-of-sight  display,  true  orientation  of  vehicles  and  gun  tables,  and  pairing  of 
firing  events  and  artillery  impact  areas.  This  type  of  system  has  applicabUity  for  improving 
feedback  and  training  in  almost  any  type  of  distributed  interactive  simulation  system. 

Army  battalions  go  to  the  National  Training  Center  (NTC)  every  18  months  for 
field  training  exercises  against  the  resident  opposition  force.  The  NTC  is  an  instrumented 
range  that  provides  both  objective  performance  data  and  evaluations  by  Observers/ 
Controllers  to  assess  unit  performance  during  exercises  for  use  in  after-action  reviews. 
The  Army  Research  Institute  examined  the  training  undertaken  by  seven  brigades  (2700 
soldiers)  in  the  six  months  before  arriving  at  the  NTC  in  order  to  identify  key  factors  that 
led  to  success  or  failure  in  the  exercises  (Keesling,  Ford  et  al.,  1992).  Strong,  positive 
correlations  were  found  for  miles  driven  in  training  (a  surrogate  measure  for  the  amount  of 
training)  and  performance  at  the  NTC;  the  correlation  was  0.68  for  force-on-force  offensive 
missions  and  0.80  for  live  fire  defensive  missions.  There  was  a  high  correspondence 
between  Observer/Controller  subjective  ratings  and  objective  casualty  exchange  ratios.  In 
general,  brigades  that  performed  better  at  the  NTC  managed  their  training  better, 
concentrated  their  training  on  the  more  critical  mission  essential  tasks  (rather  than  on  the 
entire  list  of  tasks),  more  consistently  followed  the  prescribed  Principles  of  Training  (Field 


Manual  25-100),  trained  as  combined  arms  and  Service  teams  and  used  performance- 
oriented  training  for  assessment  and  review. 

FINDINGS 

The  current  development  of  Distributed  Interactive  Simulation  systems  dedicated  to 
training  has  estimated  program  budgets  that  total  about  $4  billion.  This  includes  CCTT, 
AVCATT,  STOW,  MDT2  and  National  Guard  Training.  These  funds  are  required  to 
develop  the  hardware  and  software  needed  to  provide  large-scale  combat  environments 
before  training  can  occur  and  to  evaluate  the  effectiveness  of  training  by  these  means,  as 
well  as  for  procurement,  if  these  systems  are  shown  to  be  cost-effective.  Such  evaluations 
will  be  novel,  large-scale,  complex,  and  labor-intensive  efforts,  both  for  the  personnel 
needed  to  participate  as  subjects  in  these  tests  as  well  as  for  those  needed  to  conduct  the 
tests  and  evaluate  the  results.  Apart  from  their  size,  the  evaluations  will  be  concerned 
specifically  with  determining  the  effectiveness  of  large  forces  in  simulated  combat,  a 
difficult  and  not  well  developed  art.  It  is  important  that  planning  and  pre-testing  of 
methods  of  evaluation  and  the  development  of  alternative  training  strategies  to  be  used  to 
talfPi  advantage  of  distributed  interactive  simulation  be  undertaken  promptly. 

We  were  not  able  to  review  the  joint  training  and  exercise  program.  The  Multi- 
Service  Training  Testbed  (MDT2),  supported  by  the  Defense  Simulation  and  Modeling 
Office  and  the  Services,  appears  at  present  to  be  the  only  intensive  program  concerned  with 
the  effectiveness  of  methods  of  Joint  training.  The  initial  effort  requires  establishing  the 
Testbed  and  no  results  related  to  methods  of  training  are  available  at  present.  The  STOW 
and  National  Guard  programs  can  also  contribute  to  Joint  training.  Some  Service  efforts 
related  to  large-scale  collective  training  are  undoubtedly  relevant  to  Joint  training  but  no 
efforts  are  known  that  may  extend  Service  efforts  to  Joint  training. 


VI.  TECHNOLOGY  RELEVANT  TO  SIMULATION 

AND  TRAININGi 


In  this  chapter,  we  identify  key  issues  in  current  technology  that  are  critical  to  the 
development  and  effective  use  of  simulation  and  training  in  the  Department  of  Defense. 
Given  the  current  emphasis  placed  on  readiness  and  training  by  the  Secretary  of  Defense 
and  the  Joint  Staff,  it  is  not  surprising  that  major  attention  is  being  given  to  technology 
relevant  to  distributed  interactive  simulation,  which  is  widely  seen  as  an  affordable  way  to 
supplement  joint  exercises  and  readiness.  Thus,  the  following  technologies  are  critical  and 
significant  to  the  effective  performance  of  DIS: 

Networks 

Semi- Automated  Forces  (SAFOR) 

Terrain  and  Environment 
Range  Instrumentation 
Dismounted  Combatants 
Interfacing  to  Virtual  Environments 
Collective  Training 

This  study  did  not  investigate  all  technologies  that  may  apply  to  Simulation  and 
Training;  rather,  it  looked  at  those  technologies  whose  development  is  critical  to  its  rapid 
progress.  Recent  briefing  material^  and  available  expertise  provided  the  main  guidance  for 
selecting  these  particular  technologies.  Project  reports  and  briefing  materials  that  could  be 
acquired  were  used,  as  appropriate. 

Our  review  starts  by  identifying  the  areas  of  technology  to  be  examined.  For  each 
such  area,  we  discuss  the  most  pertinent  issues  and  present  a  table  that  shows  examples  of 
important  ongoing  projects.  Projects  are  classified  as  major  if  they  are  funded  at  over 
$2  million  a  year,  otherwise  as  minor  efforts.  The  emphasis  that  a  project  places  on  each 


1  Assistance  of  the  following  individuals  who  provided  information  used  in  this  chapter  is  gratefully 
acknowledged:  Peter  S.  Brooks,  Edward  A.  Feustel,  J.  Dexter  Fletcher,  Randy  L.  Garrett,  and  Anil  N. 
Joglekar. 

2  Advanced  Distributed  Simulation  Update  Briefing,  Institute  for  Defense  Analyses,  January  1994. 


issue  in  its  field  is  rated  as  a  major  focus  (dark  shading)  or  a  minor  focus  (light  shading). 
Then,  we  summarize  the  status  of  work  in  each  area  and  provide  conclusions  and 
recommendations.  All  dollar  estimates  are  approximations. 

NETWORKS 

Today’s  network  technologies  cannot  meet  envisioned  needs  for  real-time, 
distributed  simulation.  ARPA  has  provided  the  following  comparison  of  current  and 
envisioned  network  usage  (Figure  VI- 1): 


Today’s  Usage 

Tomorrow’s  Usage 

Small  numbers  of  large-sized  packets 

File  transfer  (latency  tolerant) 

Computer-computer  communication 
(Ethernet  optimal- 1400  Bytes) 

Performance  severely  penalized  for  other 
than  standard 

Connectivity  worldwide  to  reduce  overall 
costs  (connection-less  vice  point-to- 
point) 

Broadcast  mode  only 

Very  large  numbers  of  small-sized  packets 

Real-time  simulation  (DIS  protocol 
-250  Bytes) 

Video  teleconferencing  and  voice  traffic 

Connection-like  sessions  in  connection¬ 
less  environment 

Large  bandwidth  requirements  with 
bandwidth  reservation 

Widely  distributed  devices  requiring  larger 
WAN  bandwidth 

Large  numbers  of  members  participating  in 
many  sessions 

Figure  VI-1.  Comparison  of  Current  and  Future  Usage  of  Networks 
(Source:  Defense  Simulation  Internet  Briefing,  ARPA/ASTO,  June  1993) 


The  underlying  problem  is  one  of  scaleability;  this  is  well  recognized,  as  evidenced 
by  the  recent  ARPA-sponsored  scaleability  peer  review.^  The  issues  that  must  be 
considered  with  respect  to  scaleability  include  the  following. 

Bandwidth.  A  major  problem  occurs  when  the  number  of  simulation  entities 
exceeds  the  capacity  of  (non-intelligent)  local  area  network  adapters.  Unless  backbone 
bandwidth  can  be  delivered  to  every  workstation,  the  amount  of  information  that  can  be 
transmitted  must  be  substantially  limited.  Additionally,  there  is  a  lack  of  validated 
methodologies  for  determining  needed  bandwidth  for  a  given  exercise. 


3  Cheriton,  David  R.,  Dale  B.  Henderson,  Duncan  C.  Miller,  David  L.  Mills,  Stuart  D.  Cheshire,  Randy 
L.  Garrett,  and  Julia  A.  Loughran,  Scalability  Panel  Review,  IDA  Document  D-1451,  October  1993 
(draft). 


Latency.  Latency  is  determined  by  operating  conditions  in  exercises  and  is 
constrained  by  physical  laws,  not  subject  to  advances  in  technology.  Current  security 
mechanisms  place  an  additional  delay  that  results  in  a  latency  considerably  greater  than  the 
maximum  of  300  msec  for  application-to-application  communication  set  by  the  Communi¬ 
cation  Architecture  for  Distributed  Interactive  Simulation  (CADIS). 

Security.  This  is  a  critical  issue  that  significantly  impacts  all  other  technology 
issues.  Current  packet-oriented,  E3,  security  mechanisms  support  less  than  700  kbps. 
Mechanisms  that  exploit  the  potential  of  CX:3  (~45  Mbps)  are  needed.  Guidance  in 
determining  classification  of  exercises  is  also  needed. 

Interoperability.  This  chiefly  addresses  the  integration  of  live,  virtual,  and 
constructive  simulations,  as  well  as  linking  simulations  of  the  same  type. 

Multicasting.  Multicasting  is  required  to  help  reduce  the  volume  of 
communications  between  geographically  dispersed  participants  in  simulated  exercises; 
dynamic  management  of  multicast  group  members,  potentially  participating  in  multiple 
exercises  simultaneously,  is  also  required.  Currently,  the  dynamic  memory  of  interface 
adapters  and  security  devices  each  support  only  1  to  16  groups,  while  over  4,000  are  likely 
to  be  needed  for  a  realistic  synthetic  battlefield. 

Network  and  Exercise  Management.  The  major  problem  in  managing  a 
network  is  dealing  with  the  various  local  area  network  domains  over  a  wide  area  network. 
Exercise  management  has  to  support  the  initialization,  synchronization,  checkpointing,  and 
termination  of  exercise  entities.  There  is  a  lack  of  experience  in  dealing  with  the  orders  of 
magnitude  encountered  here. 

Focus  of  Network  Efforts 

Information  on  on-going  efforts  was  obtained  through  review  of  relevant  task 
descriptions  and  discussions  with  people  associated  with  each  effort  Several  committees 
and  infrastructure  organizations  are  concerned  with  networks. 

The  Defense  Modeling  and  Simulation  Office  (DMSO)  has  established  a  Networks 
Infrastructure  Task  Force  to  guide  research  and  development  in  the  network  area.  The 
charter  of  this  group  requires  it  to: 

Examine  the  network  infrastructure  required  for  Modeling  and  Simulation 
(M&S)  and  recommend  to  DMSO  products,  processes,  and  projects  needed  to 
address  shortfalls. 


Focus  on  wide  area  network  services  and  capacity  needed  for  growth  and 
impact  of  transition  to  fee-for-service. 

Develop  a  transition  and  integration  model/roadmap  for  the  distributed 
networking  environment. 

Identify  the  community  served  by  the  network. 

Facilitate  policy,  standards,  consensus  building  relating  to  networks,  and 
synergy  with  the  DIS  protocol  development 

ARPA  and  the  Defense  Information  Systems  Agency  (DISA)  have  formed  a  Joint 
Program  Office  (IPO)  for  developing  the  Defense  Simulation  Internet  (DSI)  into  a  new 
network  known  as  Defense  Information  Infrastructure  Network  (DIIN).  This  IPO  will 
establish  the  set  of  services  needed  to  support  simulation  in  the  1995-1998  time  frame. 

Comments 

A  summary  of  current  efforts  is  shown  in  Figure  VI-2.  Based  on  planned 
expenditures  for  known  efforts,  an  estimated  $100  million  will  be  spent  on  network 
development;  this  figure  is  expected  to  increase  substantially  in  forthcoming  years.  Due  to 
the  recent  increase  in  available  bandwidth,  there  has  been  much  progress  in  this  area  during 
the  last  six  months.  This  level  of  progress  is  likely  to  continue  for  several  more  months. 
Continued  support  of  this  critical  area  is  vital. 


KEY 


Funding 

Focus  of  technology 

Major:  over  $2  M  per  year 

Minor  less  than  $2  M  per  year 

KVVVVI 

None:  (blank) 

1  -  1 

Figure  Vl>2.  Selected  Efforts  on  Networks 


Several  networks  are  being  developed  independently  for  a  variety  of  applications, 
including  high-performance  computing,  simulation,  and  video  transmission.  Efforts  are 
needed  to  see  whether,  and  how,  these  networks  should  be  consolidated.  As  a  minimum, 
steps  should  be  taken  to  ensure  that  gateways  are  established  between  the  major  networks. 

Asynchronous  Transfer  Mode  (ATM)  is  a  new  low-level  network  protocol  with 
both  architectural  and  performance  implications.  Experiments  that  can  determine  its 
suitability  for  DIS  are  needed  before  a  commitment  is  made  to  this  approach.  Although 
network  management  is  being  addressed  quite  widely,  there  are  no  efforts  looking  at  the 
management  of  exercises,  on  top  of  network  management  systems.  This  issue  needs  to  be 
addressed. 

SEMI-AUTOMATED  FORCES 

Semi-Automated  Forces  (SAFOR)  refer  to  entities  on  a  battlefield  (e.g.,  tanks, 
aircraft,  trucks)  that  are  controlled  by  computer-based  algorithms  rather  than  human 
operators.  The  "semi-automated"  part  of  the  title  refers  to  the  fact  that  SAFORs  are 
monitored  by  a  human  controller  who  can  modify  their  performance.  SAFOR  forces  are 
needed  to  enlarge  the  number  of  participants  on  a  battlefield  without  adding  the  cost  of 
operating  more  manned  simulators  in  large-scale  exercises.  SAFOR  is  a  crucial 
underpinning  of  DIS.  The  heavy  resource  requirements  and  limited  numbers  of  manned 
simulators  will  continue  to  necessitate  the  use  of  SAFOR.  SAFOR  is  generally  used  to 
fulfill  the  role  of  threat  forces. 

Until  recently,  SAFOR  systems  had  changed  little  since  their  inception  in  the 
SIMNET  program.  The  first  SAFOR  system  provided  proof-of-principle  demonstrations 
that  established  the  feasibility  and  value  of  SAFOR  for  collective  training  applications.  The 
development  of  second-generation  systems  has  just  begun.  The  new  systems  address 
broader  communities,  a  wider  scope  of  applications,  and  the  need  for  increased  validity  in 
the  behavior  of  SAFOR  entities.  Key  issues  in  the  use  of  SAFOR  are  the  following. 

Behavioral  Realism.  This  addresses  how  SAFOR  use  the  terrain,  respond  to 
environmental  influences,  and  provide  a  credible,  as  well  as  realistic,  representation  of 
tactics  and  procedures  that  might  be  used  by  an  opposing  (or  friendly)  force.  Improvement 
of  behavioral  realism  is  needed  to  extend  the  scope  of  operational  training  and  increase  the 
validity  of  SAFOR  activities.  The  chief  difficulty  lies  in  developing  improved  models  to 
represent  more  appropriate  human  behaviors  than  the  current  SAFOR  can  represent.  To 


enhance  realism,  SAFOR  vehicles  should  also  learn  how  to  counter  the  tactics  that  live 
opponents  (in  this  case,  friendly  forces)  are  using  against  it. 

Higher  Echelon  Operations.  Current  SAFOR  DIS  models  can  operate  weapon 
systems  up  to  the  level  of  company  command.  Higher  level  operations  are  needed  to 
support  a  broader  range  of  exercises.  Demonstrations  that  include  higher  echelon 
command  centers  staffed  with  exercise  participants  are  emerging,  but  semi-automated 
support  is  required  to  fully  exploit  this  concept.  One  of  the  major  difficulties  lies  in 
providing  the  necessary  operational  environments. 

Command  and  Control  Across  Echelons.  This  issue  addresses  the  need  to 
(1)  link  the  existing  aggregate  level  command  and  control  models  into  the  DIS  framework 
and  (2)  to  link  the  command  level  staff  with  entity  level  representations.  This  new  area  is 
just  starting  to  be  investigated. 

Computational  Requirements.  SAFOR  was  introduced  as  an  economic  means 
of  filling  out  the  synthetic  battlefield.  However,  computational  requirements  increase  as  the 
complexity  of  SAFOR  entities  incorporate  greater  behavioral  realism  and  can  engage  in  a 
broader  range  of  applications.  This  increases  the  cost  of  SAFOR  forces  and  the  economics 
of  more  advanced  SAFOR  entities. 

Evaluation  Methods  and  Standards  for  Behavior.  There  is  a  continuing 
need  to  compare  the  behavior  of  SAFOR  in  different  applications  of  live  and  constructive 
simulations.  Current  efforts  have  focused  on  the  evaluation  of  algorithmic  components  of 
simulations  and  have  not  addressed  the  comparison  of  battlefield  operational  behavior  of 
simulations  working  together.  Empirically  based  studies  are  needed  to  develop  evaluation 
methods  and  standards  for  SAFOR  behavior. 

Focus  of  SAFOR  Efforts 

Information  about  on-going  efforts  shown  in  Figure  VI-3  was  obtained  through 
review  of  relevant  task  descriptions'*’^’®  and  discussions  with  people  associated  with  each 
effort.  A  series  of  STRICOM/DMSO-sponsored  reviews  of  the  state  and  direction  of 


4  Synthetic  Forces  Briefing,  ARP  A,  ASTO  Program  Review,  August  1993. 

5  STRICOM  Simulation  and  Training  Technology  Base,  Significant  Events  Report,  STRICOM, 
July-September  1993. 

6  BBS  and  SIMNET  Functional  Prototype  Project  Update,  September  1993. 
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SAFOR  technology  have  been  conducted.  The  findings  from  the  most  recent  of  these 
surveys”^  address  the  following  points: 

•  Characteristics  of  existing  SAFOR  architectures  that  should  be  incorporated 

into  future  development  efforts. 

Problems  and  issues  that  have  not  yet  been  addressed,  for  example,  lack  of  a 
balanced  representation  of  joint  operations. 

•  Enabling  technologies,  for  example,  general  computing  technologies. 

In  general,  the  sponsorship  for  work  to  improve  SAFOR  should  be  broadened.  A 
workshop  could  develop  an  organized  approach  to  defining  SAFOR  requirements  and 
monitoring  advances  in  technology  with  respect  to  investments. 
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Figure  VI-3.  Selected  Efforts  on  Semi-Automated  Forces 


1993  DMSO  Survey  of  Semi-Automated  Forces,  July  30, 1993. 
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Comments 


Based  on  the  current  and  projected  funding  for  efforts  in  this  area,  an  estimated 
$25  million  will  be  spent  this  year.  MODSAF  (Modular  SAFOR),  and  related  ARPA 
programs,  will  be  the  main  consumer  of  these  resources.  The  resources  invested  in 
SAFOR  development  to  date  have  resulted  in  useful  progress  being  made.  Considering  the 
crucial  role  that  SAFOR  plays  in  DIS,  continued  investment  is  justifiable. 

The  programs  being  conducted  in  this  area  are  not  closely  linked,  and  some  relevant 
activities  are  being  accomplished  in  different  areas;  e.g.,  this  needs  to  be  examined  to 
ensure  that  related  programs  can  appropriately  leverage  off  each  other.  Therefore,  one 
strategic  recommendation  is  to  build  closer  coordination  between  the  organizations 
undertaking  SAFOR  programs.  In  addition,  the  needs  of  all  potential  user  groups  must  be 
considered  in  order  to  ensure  that  the  broadest  possible  use  can  be  made  of  the  developed 
SAFOR  technology.  In  addition  to  training,  the  potential  user  community  includes  those 
with  command  and  acquisition  orientations.  More  effort  is  needed  on  evaluation  methods 
and  standards  for  the  behaviors  exhibited  by  SAFOR. 

TERRAIN  AND  ENVIRONMENT 

Modeling  and  simulation  needed  to  support  effective  training  is  dependent  on  the 
location  and  environment  of  the  scenario,  and  may  need  high  fidelity  physics  and 
engineering  models  to  provide  a  synthetic  environment  that  includes  time-and-space- 
varying  information  about  the  terrain,  atmosphere,  atmospheric  backgrounds,  oceans,  and 
near-space  with  which  vehicles,  weapons,  sensors,  and  communication  must  interact.  The 
following  factors  dominate  the  technology  needed  to  accurately  represent  the  terrain  and 
environment  of  an  engagement 

Representation.  The  representation  of  terrain  and  environmental  data  should 
incorporate  such  features  as  visibility,  electro-magnetic,  and  electro-optical  propagation, 
voice  communication,  and  vehicle  movement.  It  should  be  physically  scaleable  to  adjust  to 
the  constraints  and  capabiUties  of  different  simulations  and  simulators. 

Rapid  Generation.  The  generation  of  detailed  terrain  databases  is  currently  a 
time-consuming  process.  Means  of  rapid  database  generation  are  needed  to  aUow  the  timely 
support  of  unpredictable  training  requirements.  Standards  for  protocols  that  permit  the  use 
of  existing  databases  are  also  needed. 


Dynamic  Terrain.  The  current  synthetic  battlefield  does  not  support  modification 
of  the  environment  in  response  to  simulated  events.  For  example,  the  effects  of  explosions 
on  the  land  surface,  and  the  effects  of  chemical  weapons  on  atmospheric  conditions,  or 
actions  taken  by  combat  engineers  to  place  bridges  or  create  barriers,  cannot  be 
represented.  The  capability  to  support  a  changeable  environment  in  real  time  is  needed. 

Computation  and  Connectivity.  There  is  a  need  for  real-time  connectivity. 
Detailed  representation  of  changing  terrain  and  environment  poses  substantial 
computational  demands;  efficient  support  for  Computer  Image  Generation  (CIG)  is  also 
needed;  networking  of  heterogeneous  CIGs  must  also  be  addressed. 

Standards.  Standards  are  needed  to  simplify  and  facilitate  the  integration  of 
information  on  the  synthetic  physical  environment.  These  standards  should  address  such 
items  as  database  structures,  transfer  formats,  and  Protocol  Data  Units  (PDUs).  Their 
development  must  be  coordinated  with  existing  DIS-related  efforts  towards  standardization. 

DMSO  has  identified  goals  for  representation  of  the  physical  environment. 
Although  the  technology  issues  represented  are  a  composite  of  those  previously  discussed. 
Figure  VI-4  provides  a  summary  of  the  desired  advances  in  the  terrain  and  environment 


area. 


Aspects  of  the 
Environment 

Mid-Term  Goals 
(1995-1999) 

Long-Term  Goals 
(2000-2008) 

Data  Bases 

Network  accessibility  and 
portability  of  existing  data 
bases  across  all 
environmental  domains 

High-speed,  high-resolution 
coverage  of  the  geospace 
with  integrated  and  correlated 
treatment  of  the  whole 
environment 

Models 

Adaptation  and  application  of 
existing  models  to  DIS 

Synthesis  of  fast,  full- 
spectrum,  high-fidelity  all 
weather  models  of  the 
changing  environment  and  its 
effects  on  systems 
performance 

Visualization 

Display  of  selected 
environmental  components 
and  features,  with  limKed 
update  capability  and 
scaleability 

Accurate,  realistic  and  fully 
scaleable  displays  of  the 
environment,  with  real-time 
interaction 

Figure  VI-4.  Goals  for  Representation  of  the  Physical  Environment  in 
Distributed  Interactive  Simulation 


Focus  of  Terrain  and  Environment  Efforts 

Information  on  on-going  efforts  shown  in  Figure  VI-5  was  obtained  through 
review  of  relevant  task  descriptions^-^  and  discussions  with  people  associated  with  each 
effort. 
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Figure  VI-5.  Selected  Efforts  on  Terrain  and  Environment 


Comments 

Based  on  planned  expenditures  for  known  efforts,  an  estimated  $100  million  will 
be  spent  this  year.  Similar  amounts  are  expected  to  be  spent  in  the  next  couple  of  years. 

While  issues  with  respect  to  representing  terrain  and  environment  in  simulation  are 
not  new,  work  looking  at  these  issues  in  the  context  of  large-scale  modeling  and  simulation 
for  DIS  has  just  begun.  Accurate  representation  of  terrain  and  environment  is  critical  to  the 
success  of  distributed  interactive  simulation  for  all  purposes,  and  progress  is  expected  to 
occur  over  the  next  few  years.  The  rapid  generation  of  terrain  and  environment  is  needed 
particularly  to  respond  to  terrorists  and  other  no-warning  hostile  events  that  require  an 
almost  immediate  military  response.  Authoritative  representation  of  terrain  and 


8  E2DIS  Project  Development  Plan,  prepared  for  DDR&E  and  DMSO,  June  1993. 

9  DMSO  FY94  Focused  Call  and  Team  Assessment,  Project  94-026. 


environment  is  needed  for  planning  at  all  levels,  including  joint  operations.  There  is  also  a 
need  to  make  certified  and  accredited  representations  available  to  the  modeling  and 
simulation  community. 

Major  technical  concerns  relate  to  the  management  rather  than  to  the  development  of 
new  technology.  In  particular,  there  is  a  critical  need  for  terrain  and  environment  standards 
and  supporting  policy.  The  availability  of  appropriate  standards  will  support  and 
encourage  developing  the  necessary  compatibility  between  different  terrain  and 
environment  databases. 

RANGE  INSTRUMENTATION 

Most  current  field  exercises  are  oriented  towards  a  single  Service  and  are  often 
limited  in  scope  because  of  cost  and  environmental  factors.  The  current  inability  to  include 
Division  and  Corp  level  echelons  in  DIS  and  the  limited  number  of  participants  are  just  two 
examples  of  factors  that  impact  the  operational  realism  of  these  exercises.  The  ability  to 
overcome  these  limitations  by  linking  live  simulations  with  virtual  and  constructive 
simulations  is  crucial  to  supporting  today’s  vision  of  exercises  and  mission  rehearsal.  By 
supplementing  field  exercises  with  simulation-based  unit  training,  both  the  span  and  joint 
character  of  training  for  combat  could  be  significantly  improved;  in  this  way,  data  needed 
for  Verification,  Validation  and  Accreditation  (VV&A)  of  models  and  simulations  could 
also  be  developed.  The  need  to  facilitate  this  linkage  underlies  many  of  the  issues 
discussed  below. 

Compatibility  and  Interoperability.  Several  concerns  need  to  be  addressed. 
One  aspect  is  the  compatibility  between  levels  of  detail  in  interactive  live,  virtual,  and 
constructive  simulations.  Another  is  the  difficulty  of  integrating  activation  of  entities  within 
a  given  visual  range.  The  compatibility  and  interoperability  of  older  systems  is  also  a 
concern.  Full  interoperability  requires  that  the  issues  of  standards,  latency,  security, 
environmental  compatibility,  and  time  synchronization  be  resolved. 

Adequate  Terrain  and  Environment  and  Threat  Fidelity.  The  linkage  of 
live  simulations  with  real  and  virtual  simulations  imposes  more  precise  fidelity 
requirements.  Digital  terrains  are  a  necessity,  along  with  a  useful  representation  of 
environmental  conditions.  The  level  of  fidelity  of  computer-generated  threats  is  also 
critical.  Work  is  just  beginning  in  this  area. 

Embedded  Instrumentation.  The  instrumentation  needed  to  record  perform¬ 
ance  in  field  exercises  is  currently  independent  equipment  added  onto  personnel  and 
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platforms.  The  penalties  of  weight,  power,  and  cost  could  be  mitigated  by  integrating 
instrumentation  systems  with  existing  navigation  and  communications  systems  on  various 
platforms. 

Multi-Level  Security  (Encryption).  The  Air  Force  and  Navy  require  that  data 
transmitted  on  ranges  be  encrypted  to  allow  the  inclusion  of  advanced  equipment  in  live 
exercises,  whereas  Army  instrumentation  cannot  be  classified.  Hence,  interoperability 
between  these  Services  on  ranges  requires  multi-level  security.  With  the  advent  of  the  B-2 
and  F-22  aircraft  in  the  next  few  years,  there  is  an  urgent  need  for  this  capability.  Range 
instrumentation  systems  add  difficult  security  requirements  to  DSI  pertaining  to  size, 
weight,  air-worthiness,  and  remote  over-the-air  rekeying  and  zeroizing. 

C3I  Representation  and  Support.  There  is  a  need  to  define  the  C3I 
information  that  should  be  captured  to  support  concurrent  virtual  and  live  exercises.  This 
information  would  provide  a  mechanism  for  integrating  different  ranges,  in  addition  to 
integrating  the  different  types  of  simulation.  One  of  the  questions  to  be  addressed  is 
whether  C3I  information  should  be  recorded  outside  the  range  instrumentation  system  or 
DIS,  or  as  another  type  of  DIS  entity.  Fidelity,  latency,  and  bandwidth  are  also  concerns. 

Standards.  Standards  in  data  types,  data  formats,  data  update  rates,  operational 
frequency,  and  network  protocols  are  critical  to  the  linkage  of  live  simulations  with  virtual 
and  constructive  simulations  across  the  DIS  network.  The  DIS  standard  addresses  only 
some  of  these  concerns;  challenges  in  the  area  of  data  rate  and  bandwidth  need  to  be 
addressed.  The  majority  of  range  instrumentation  systems  rely  on  radio  communication 
networks.  The  bandwidth  of  these  networks  is  limited  to  200-800  kbps,  as  opposed  to 
DIS  data  rates  of  1.56  Mbps  or  higher. 

Focus  of  Selected  Range  Instrumentation  E^orts 

Information  on  on-going  efforts  was  obtained  through  review  of  relevant  task 
descriptions*®’*^  and  discussions  with  people  associated  with  each  effort;  these  are 
summarized  in  Figure  VI-6. 


*®  Acquisition  Plan  for  JACTS,  No.  AP53-(X)1,  Aeronautic  Systems  Center,  Eglin  AFB,  Florida,  June 
1993. 

1 1  JADS  in  Progress  Review  by  Technical  Advisory  Board,  Kirtland  AFB,  New  Mexico,  March  1994. 
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Comments 

Based  on  current  and  projected  funding  for  efforts  in  this  area,  an  estimated 
$50  million  will  be  spent  this  year.  Of  this,  research,  development,  test  and  evaluation  on 
the  Joint  Air  Combat  Training  System  (JACTS)  and  Tactical  Combat  Training  System 
(TCTS)  will  consume  roughly  $17  million. 

The  community  is  sensitive  to  range  instrumentation  needs  and  many  discussions 
have  been  held.  Interoperability  must  be  built  into  new  systems  and  the  start  of  positive 
actions  can  be  seen  in  a  decision  to  halt  development  of  the  JACTS  and  TCTS  systems  until 
interoperability  issues  have  been  addressed.  When  it  is  too  late  to  fully  address 
interoperability  in  emerging  systems  (e.g.,  MATS),  hooks  must  be  included  to  facilitate  the 
phased  introduction  of  interoperability  over  time.  Meanwhile,  there  are  many  incompatible 
existing  range  instrumentation  systems,  representing  a  national  investment  of  some  $2 
trillion.  A  roadmap  is  needed  to  identify  and  guide  the  work  needed  to  result  in  a  set  of 
fully  interoperable  range  instrumentation  systems. 
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Figure  VI-6.  Selected  Efforts  on  Range  Instrumentation 


It  will  take  many  years  to  reach  the  goal  of  fully  interoperable  range  instrumentation 
systems.  However,  with  the  exception  of  network  bandwidth,  range  instrumentation 
issues  are  not  impacted  by  difficult  technical  challenges.  Additional  resources  can  speed  up 
progress.  The  development  of  needed  standards  is  particularly  critical. 
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DISMOUNTED  COMBATANTS 

The  role  and  importance  of  dismounted  combatants,  also  referred  to  as  individual 
combatants,  on  the  battlefield  has  changed  radically  in  recent  years.  This  has  led  to  a 
requirement  for  higher  mission  assurance  and  lower  vulnerability  for  the  individual 
combatant  The  ability  to  allow  dismounted  combatants  to  interact  with  mounted  forces  in  a 
synthetic  environment  provides  a  valuable  training  opportunity  that  responds  to  this 
requirement.  The  crucial  test  of  technology  developments  will  be  the  extent  to  which  their 
inclusion  of  dismounted  combatants  enhances  the  utility  of  DIS  for  training  field  officers, 
as  well  as  for  mission  rehearsal,  logistics  functions,  and  virtual  prototyping  and  testing  of 
new  equipment 

As  with  the  other  areas  of  technology  considered  here,  the  technology  required  to 
support  individual  combatants  is  not  unique  to  this  area.  For  example,  resolving  the 
technology  issues  associated  with  individual  combatants  in  DIS  will  resolve  many  of  the 
problems  with  instrumenting  individuals  in  live  exercises. 

Broadband  Wireless  Communications.  A  current  issue  is  the  introduction  of 
voice  communications  into  DIS.  Voice  communication  PDUs  are  expected  to  outnumber 
entity  representation  PDUs  by  a  ratio  of  3:1.  Possible  resolutions  to  the  increased 
throughput  demands  due  to  adding  voice  must  consider  some  new  factors,  such  as  the 
effects  of  latency  on  human  perception. 

Environment  Construction.  Current  simulations  are  based  on  Digital  Terrain 
and  Elevation  Data  (DTED)  Level  1,  a  level  of  detail  acceptable  for  interaction  with 
simulated  vehicles  but  not  for  individual  combatants.  The  level  of  detail  needed  for 
individual  combatants  has  not  been  specified  and  must  be  determined.  Furthermore, 
individual  combatants  require  the  addition  of  vertical  relief  to  support  the  representation  of 
cover  and  concealment 

.  Human  Perception.  Immersion  of  individuals  in  a  synthetic  environment 
introduces  many  new  concerns,  such  as  high-resolution  physical  detail  and  behavior  and 
cognition  databases.  (Interface  and  sensory  interaction  issues  are  discussed  below  in  the 
section  on  Interfacing  with  Synthetic  Environments.) 

Behavioral  Representation.  Simulation  of  individual  combatants  will  require 
icons  of  human  figures  which  realistically  portray  a  person’s  body  and  behaviors  for 
friendly,  enemy,  and  neutral  soldiers,  as  well  as  the  observer.  Actual  soldiers  must  be  able 
to  interface  with  these  icons  through  the  use  of  sensor  devices  that  provide  real-time  control 


# 
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of  the  model  and  data  collection  on  their  interactions.  Various  issues  that  need  to  be 
addressed  include  human  response  time,  resource  limits,  collision  avoidance,  multi-person 
cooperative  tasks,  and  handling  three-dimensional  acoustic  data,  as  observed  in  the  display. 

Including  Combatants  in  DIS.  Existing  DIS  draft  standards  have  not 
addressed  special  concerns  relating  to  individual  combatants  operating  in  synthetic 
environments.  Such  concerns  include  greater  fidelity  in  terrain  representation  in  which 
humans  can  perform  credibly. 

Computational  Requirements.  Support  for  individual  combatants  will  impose 
additional  computational  requirements.  This  is  due  to  the  impact  of  an  increased  bandwidth, 
and  the  need  for  more  detailed  terrain  data  and  consequent  trade-off  of  detail  for  an  area  of 
concern  while  preserving  vertical  and  horizontal  control. 

Focus  of  Selected  Individual  Combatant  Efforts 

Information  on  ongoing  efforts  was  obtained  through  review  of  relevant  task 
descriptions*^, 13, 14,15,16  and  discussions  with  people  associated  with  each  effort;  these  are 
summarized  in  Figure  VI-7. 

Comments 

Simulation  technology  needed  to  portray  dismounted  combatants  on  the  battlefield 
is  a  new  area  of  research  and  development.  The  need  for  work  in  this  area  was  highlighted 
in  General  Gorman's  paper  on  SuperTroop  (Gorman,  1990),  and  is  evident  through  the 
use  of  individual  combatant  technology  in  the  21st  Century  Land  Warrior  Top  Level 
Demonstration.  Based  on  planned  expenditures  for  known  efforts,  less  than  $10  million 
will  be  spent  on  this  topic  this  year. 

Recommendations  for  work  in  this  area  reflect  the  need  to  establish  an  infrastructure 
to  organize  and  guide  on-going  work.  An  up-front  analysis  is  needed  to  determine  how 
individual  combatant  technology  should  be  used  to  exploit  the  fullest  potential  from  on¬ 
going  and  future  work  in  this  area.  The  primary  question  to  be  answered  is:  At  what  level 


12  DMSO  FY94  Focused  Call  and  Team  Assessment,  Project  94-010. 

1 3  DMSO  FY94  Focused  CaD  and  Team  Assessment,  Project  94-07 1 . 

Project  Execution  Plan  for  Environmental  Representation  of  Urban  Terrain,  DMSO  Project  94072 
(undated). 

15  The  Integrated  Unit  Simulation  System  Briefing,  US  Army  Natick  RD&E  Center  (undated). 

16  Simulation  of  the  Individual  Combatant  Briefing,  Cardinal  Point,  Inc.,  November  1993. 
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and  with  what  amount  of  detail  should  individuals  be  introduced  in  various  types  of 
simulations  and  missions? 
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Figure  VI-7.  Selected  Efforts  on  Dismounted  Combatants  in  Simulations 

Individual  combatant  simulation  technology  has  considerable  dual-use  potential. 
Technology  that  supports  training  the  individual  warrior  has  potential  application  for  the 
training  of  many  types  of  specialized  operations,  including  fire  fighting,  special  operations, 
and  police  work.  Liaison  needed  to  exploit  this  dual-use  potential  should  be  promoted. 

INTERFACING  WITH  VIRTUAL  ENVIRONMENTS 

With  the  exception  of  limited  numbers  of  manned  simulators,  the  traditional 
interface  to  training  simulations  has  been  through  the  computer  screen.  Virtual  reality 
technology  attempts  to  provide  a  higher  bandwidth  communication  channel  between 
simulations  and  humans  by  immersing  humans  in  a  synthetic  environment.  Though  the 
practical  capabilities  of  this  advanced  interface  technology  are  still  uncertain,  many  believe 
that  it  offers  a  significant  potential  for  military  training.  The  issues  identified  below  reflect 
critical  concerns  for  both  screen-based  and  immersion-based  interfaces;  they  apply  to 
interfaces  for  both  operators  of  and  participants  in  simulations. 

Computer  Image  Generation  (CIG).  The  degree  of  realism  achievable  in 
computer  generated  images  is  determined  chiefly  by  interrelated  factors  such  as  polygon 
and  pixel  processing  rates,  the  number  of  moving  models,  and  the  frame  update  rate. 
Many  groups  are  already  looking  at  how  to  provide  increased  capacity  for  CIGs  at 
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affordable  costs.  Further  examination  of  perceptual  issues  is  needed  to  determine  what 
physical  capacity  is  needed. 

Displays.  Problems  with  monitors,  flat  panel  displays,  and  Liquid  Crystal 
Displays  (LCDs)  are  largely  those  of  manufacturing  and  affordability,  and  these  are  being 
addressed  by  industry.  Helmet  mounted  displays  (HMDs)  and  wrap-around  projection 
screens  are  examples  of  technologies  that  warrant  investigation  for  their  role  in  DIS.  Since 
each  type  of  display  has  some  advantages,  it  is  likely  that  future  simulation-based  exercises 
will  requite  the  use  of  multiple  types  of  displays. 

Multi-Sensory  Input/Output.  The  usefulness  and  fidelity  of  the  immersion 
experience  depends  on  the  sensory  environment  provided  by  the  interface.  Progress  has 
been  made  for  visual  and  auditory  interfaces;  progress  in  other  modes  of  interaction 
remains  minimal.  Advances  are  needed  in,  for  example,  unobtrusive  support  for 
locomotion,  tactile  and  force-feedback  interactions,  tracking  and  navigation.  The  research 
must  address  the  physiological,  psychological,  and  perceptual  issues  related  to  such  multi¬ 
modal  interfaces. 

Visualization.  How  complex  data  should  be  presented  to  a  user  is  another 
important  issue  that  needs  clarification  and  further  investigation.  For  example, 
visualization  and  spatial  interpretation  of  massive  databases,  such  as  large  air-ground  areas, 
the  human  body,  and  ocean-land  interfaces  should  be  supported. 

Adaptable,  Intelligent  User  Interfaces.  Interfaces  need  to  be  adaptive  to  the 
capabilities  and  needs  of  the  different  users.  Moreover,  factors  such  as  the  wealth  of 
information  that  simulations  are  envisioned  to  make  available  and  the  increasing  need  for 
faster  decision  response  times  are  increasing  the  burden  on  simulation  operators  and  users. 
Intelligent  interface  agents  that  can  mitigate  this  burden  and  provide  some  form  of 
intelligent  training  will  be  needed. 

User  Interface  Design  Methods.  User  interface  concerns  can  no  longer  be 
considered  in  isolation  from  other  design  issues.  The  needs  and  behaviors  of  operators, 
participants,  and  users  of  collected  data  must  be  modeled  and  actively  included  in  the 
earliest  design  activities.  While  these  concerns  are  important  for  training  systems 
generally,  their  importance  becomes  crucial  when  interfacing  with  synthetic  environments. 


Focus  of  Efforts  on  Interfacing  to  Virtual  Environments 

Information  on  on-going  efforts  Avas  obtained  through  review  of  relevant  task 
descriptions  and  discussions  with  people  associated  with  each  effort.  Current  efforts  are 
summarized  in  Figure  VI-8. 
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Figure  VI-8.  Selected  Efforts  on  Interfacing  with  Virtual  Environments 
Comments 

Given  the  diverse  nature  of  on-going  work,  and  the  multitude  of  players,  it  was  not 
possible  to  estimate  the  level  of  effort  being  invested  in  interfacing  to  virtual  environments. 
Progress  varies  by  area.  While  advances  have  been  made  in  display  resolution,  work  on 
touch  and  feel  (haptic)  interaction  has  been  less  successful.  The  technology  as  a  whole  is 
advancing,  but  little  is  known  about  the  worth  of  haptic  interfaces  on  training  and  military 
performance. 

The  virtual  reality  (VR)  aspects  of  this  technology  have  been  widely  publicized  in 
recent  years.  There  have  been  numerous  Service  sponsored,  and  other  Government  agency 
sponsored,  working  groups  looking  at  VR-related  issues.  Interest  in  the  technology  is  not 
limited  to  the  United  States;  for  example,  the  Tri-Service  VR  Working  Group  for  Training 


AppUcations  (UTP-2  Military  Training  Panel  of  TTie  Technology  Cooperation  Program)  has 
been  fostering  collaboration  on  this  topic  among  the  five  English-speaking  countries.  In 
1992,  the  Federal  Coordination  Committee  for  Science,  Engineering  and  Technology 
(FCCSET)  established  a  Task  Group  under  the  Subcommittee  on  High  Performance 
Computing  and  Communications  and  Information  Technology  (HPCCIT)  to  assess 
Government  interests  and  activities  with  this  technology. The  National  Academy  of 
Sciences  proposed  an  effort  to  develop  a  national  agenda  for  VR  research. 

The  diverse  Government  groups  active  in  this  area  are  largely  working 
independently.  There  is  a  serious  possibility  that  several  independent  development 
programs  are  not  coordinated.  In  order  to  avoid  unnecessary  duplication  of  work,  and  the 
development  of  incompatible  systems,  the  DoD  must  determine  how  to  coordinate  virtual 
environments  interface  work  within  the  DoD  as  a  whole  and,  preferably,  within  the  entire 
Government. 

In  functional  teims,  the  technology  associated  with  virtual  reality  is  treated  here  as  a 
segment  of  ModeUng  and  Simulation.  Despite  the  large  number  of  on-going  efforts 
looking  at  many  possible  applications,  there  has  been  little  work  looking  at  the  specific 
application  of  virtual  environments  interface  technology  to  military  modeling  and 
simulation.  The  DoD  needs  to  determine  its  near-term  and  long-term  needs  in  order  to 
focus  on-going  and  future  work  that  it  supports.  Except  for  Computer  Image  Generation 
(CIG)  and  displays,  work  in  this  area  is  in  its  infancy  and  needs  careful  consideration  and 
guidance.  CIG  issues  are  being  effectively  addressed  by  industry.  Specific  technical 
recommendations  for  helmet  mounted  displays  (HMDs)  include  the  following: 

Provide  engineering  specification  for  devices  to  allow  definition  of  the 
spectrum  of  fidelity. 

Develop  test  equipment  to  facilitate  quality  assurance  of  devices. 

Assess  impact  of  VR  sickness  on  training  effectiveness  and  operational 
performance. 

Develop  rules  that  promote  easy  adaptation  and  re-adaptation  to  VR  devices. 

Similar  concerns  will  arise  for  other  new  input/output  devices  as  their  development 
progresses.  In  general,  more  attention  should  be  paid  to  integrating  human  factors 


Virtual  Reality  Technology:  A  Report  of  the  Task  Group  on  Virtual  Reality  to  the  High  Perfonnmce 
Computing  and  Communications  and  Information  Technology  (HPCCIT)  Subcommittee,  The 
FCCSET  Cranmittee  on  Physical  Mathematical  and  Engineering  Sciences,  August  1993. 


engineering  considerations  into  on-going  efforts.  Further  examination  of  perceptual  issues 
is  needed  to  determine  required  CIG  capacity. 

COLLECTIVE  TRAINING 

Live,  virtual,  and  constructive  simulations  are  major  tools  used  for  military  training 
and  their  utility  for  developing  and  assessing  joint  readiness  is  likely  to  increase.  While  the 
availability  of  such  tools  is,  of  course,  crucial,  so  is  an  understanding  of  what  to  teach, 
how  to  teach  it,  when  to  teach  it,  and  whether  that  teaching  was  effective.  Pertinent  issues 
of  technology  needed  to  support  collective  training  are  discussed  next. 

Performance  Measurement.  The  contribution  that  training  makes  to  military 
readiness  cannot  be  assessed  accurately  in  the  absence  of  performance  measurement 
methods.  There  are  many  methods  for  measuring  the  performance  of  individuals,  but  there 
are  few  methods  for  assessing  the  performance  of  units  or  collective  groups.  The  absence 
of  agreed  definitions  of  training  objectives  and  of  valid  measures  makes  this  a  difficult  but 
no  less  crucial  deficiency  to  address.  The  incorporation  of  performance  measures  at  every 
level  of  simulation  will  be  the  key  to  getting  accurate  measures  of  readiness  at  all  levels, 
i.e.,  individual,  unit,  and  joint  readiness. 

Instructional  Strategy.  This  is  a  need  for  determining,  for  example,  whether 
apprenticeship  or  Socratic  training  methods  are  more  suitable  for  training  on  particular  unit 
tasks.  There  are  approaches  for  choosing  between  different  instructional  strategies  for 
training  individuals,  but  there  is  a  lack  of  validated  methods  for  unit  training.  In  particular, 
there  is  a  lack  of  understanding  on  how  best  to  use  combinations  of  live,  virtual,  and 
constructive  simulations  in  a  complementary  manner  for  training. 

Defining  Critical  Scenarios.  The  Services  and  the  Joint  Staff  have  developed 
lists  of  essential  military  tasks. Methods  for  generating  scenarios  for  critical  essential 
tasks  are  needed  to  ensure  the  relevance  of  simulation  exercises  to  training  and  military 
requirements  at  the  highest  levels. 

Cost-Effective  Training  Mix.  There  has  been  some  interest  in  defining  the 
appropriate  mix  of,  for  example,  simulation  and  live  training,  at  the  individual  level  but 
little  at  the  unit  level.  Determining  unit-level  cost-effectiveness  is  particularly  difficult  since 
the  set  of  missions  that  must  be  supported  varies  widely,  e.g.,  from  major  regional 


^  *  Universal  Joint  Task  List,  MCM-147-93,  The  Joint  Staff,  25  October  1993. 


contingencies  to  operations  other  than  war;  unit  training  cost  data  are  not  easily  identifiable 
in  Operation  and  Maintenance  accounts. 

Skill  Decay  and  Maintenance.  Once  again,  more  is  known  about  skill  decay 
and  maintenance  for  individuals  than  for  units.  Here,  the  ability  to  measure  performance  is 
a  prerequisite  to  development  of  training  strategies  for  crews  and  units. 

Focus  of  Training  Efforts 

Information  on  on-going  efforts  was  obtained  through  a  review  of  relevant  task 
descriptions^^’^®’^^  and  discussions  with  people  associated  with  each  effort;  this  work  is 
summarized  in  Figure  VI-9. 
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Figure  VI-9. 


Selected  Efforts  on  Collective  Training 


19  DMSO  FY93  FCP  Program  Status  Summary  Reports,  1993,  Project  93-025. 

20  MDT2  Project  Development  Plan,  DMSO  (undated). 

21  Series  of  NTSC  Technical  Reports. 
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Comments 

Based  on  current  funding  for  the  cited  efforts,  an  estimated  $15  million  will  be 
spent  this  year  in  this  area  as  a  whole.  The  Multi-Service  Distributed  Training  Testbed 
(MDT2)  effort  will  consume  nearly  half  these  resources. 

The  technology  needed  to  support  collective  training,  i.e.,  SIMNET  and  its 
derivatives,  has  been  well  established.  But  there  is  a  recognized  and  critical  need  for 
training  strategies  that  can  take  advantage  of  large-scale,  distributed  interactive  simulation 
technology.  Some  enabling  technologies,  such  as  cognitive  modeling,  still  have  limited 
capabilities.  What  is  lacking  is  the  ability  to  apply  these  understandings  to  such  issues  as 
performance  measurement  and  instructional  strategy  for  training  collectives,  up  to  the  level 
of  joint  operations. 

There  has  been  insufficient  attention  to  this  area.  Without  any  real  rapid  progress, 
the  lack  of  adequate  training  technology  may  severely  limit  the  utility  and  use  of  DIS  for 
training.  It  is  uncertain  whether  spending  more  money  can  accelerate  resolution  of  the 
identified  technologies.  A  defined  set  of  unit  training  objectives  must  be  established  and 
agreed  upon.  Demonstrations  may  provide  a  useful  mechanism  for  forcing  consideration  of 
the  pertinent  issues  and  helping  to  identify  these  objectives.  The  basis  for  this  exists  in  the 
Universal  Joint  Task  List  and  the  Joint  Military  Essential  Task  Lists  of  the  CINCs.  These 
task  lists,  with  their  conditions  and  standards,  should  be  used  to  drive  R&D  and  the 
application  of  training  technology. 

The  issues  discussed  here  are,  in  general,  subordinate  to  the  issue  of  instructional 
strategy  for  large  combat  units  and  this  is  the  most  important  issue.  In  addition, 
performance  measurement  is  needed  to  support  R&D  for  all  the  other  discussed  issues. 

FINDINGS 

A  review  of  technologies  critical  to  simulation  and  training  examined  R&D  on 
distributed  interactive  simulation  in  areas  concerned  with  networks,  semi-automated  forces, 
terrain  and  environment,  range  instrumentation,  individual  combatants,  virtual 
environments,  and  training  technology.  Areas  that  receive  major  funding  ($100  million  a 
year  or  more)  are  networks,  terrain,  and  environment  and  range  instrumentation.  Within 
these  areas,  increased  attention  should  be  given  to  develop  the  Asynchronous  Transfer 
Mode  (ATM)  protocol  for  DIS,  support  for  exercise  management,  compatibility  between 
terrain  and  environment  databases,  radio-frequency  network  bandwidths  for  range 
instrumentation,  and  communication  standards.  Less  support  is  being  given  to  methods  of 
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training,  design  &nd  use  of  individual  combatants  in  DIS  and  SAFOR.  There  is  a  need  for 
development  of  methods  of  evaluating  performance  in  all  areas,  clarification  of  appropriate 
applications  of  individual  combatants  (including  dual-use  potential)  in  DIS,  and  effective 
ways  to  use  and  measure  the  training  effectiveness  of  DIS  systems. 
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VII.  DISCUSSION 


The  current  draw-down  of  about  30  percent  in  force  structure  and  manpower  leads 
to  serious  problems  concerning  the  ability  of  the  armed  forces  to  accomplish  their  assigned 
missions.  At  the  same  time,  missions  have  changed  significantly  from  those  associated 
with  a  nuclear-dominated,  bi-polar  world  to  those  associated  with  a  multi-polar  world, 
major  regional  conflicts,  small  contingencies,  and  operations  other  than  war.  Senior 
advisory  groups  that  have  recently  reviewed  the  current  missions  of  the  Department  of 
Defense  find  that  training  is  a  major  component  of  joint  readiness  and  that  distributed 
interactive  simulation  provides  an  important  way  to  provide  the  training  needed  to  maintain 
readiness  (Defense  Science  Board,  1993;  DSB  Task  Force  on  Readiness,  1994;  Training 
Readiness  in  the  DoD,  1994:  Burba,  et  al.,  1994).  With  this  background,  we  wish  to 
discuss  lessons  learned  about  simulation  and  training  derived  from  information  presented  in 
this  report.  Where  problems  are  identified,  the  purpose  is  to  be  constructive  rather  than 
critical. 

Simulation  is  demonstrably  a  cost-effective  method  of  military  training.  When  it  is 
evaluated,  it  is  generally  found  to  be  as  effective  in  training  specific  skills  as  the  use  of 
actual  equipment  would  be;  it  saves  student  time  and  costs  less  to  acquire  and  use  than 
actual  equipment.  Savings  due  to  the  use  of  simulators  for  training  can  amortize  their  cost 
in  periods  of  one  to  four  years.  This  is  a  conservative  estimate  that  does  not  include  the 
costs  of  wear  and  damage  to  actual  equipment  used  for  training  or  of  acquiring  additional 
operational  equipment  needed  to  satisfy  training  requirements.  Very  preliminary  findings, 
based  on  tests  with  SIMNET,  suggest  that  distributed  interactive  simulation  should  be 
effective  for  combat  training  of  armor  forces.  Compared  to  large-scale  field  exercises,  DIS 
will  obviously  provide  large  cost  savings  as  well  as  permit  joint  training  that  cannot  be 
accomplished  in  any  other  way  because  of  the  prohibitive  costs  of  exercises  (including 
transportation,  ammunition,  and  damage  to  equipment  and  the  environment).  If  distributed 
simulation  is  used  to  test  tactics  for  use  in  combat,  or  in  preparing  troops  for  actual  combat, 
it  can  avoid  public  attention  and  protect  planning  and  security.  Confirmation  of  early 
estimates  of  the  potential  effectiveness  and  cost  of  distributed  interactive  simulation  should 
be  a  high  priority  requirement,  particularly  as  it  applies  to  joint  training  and  readiness.  The 
investment  in  simulation,  with  its  array  of  applications  and  potential  contributions  to 
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readiness,  appears  to  be  modest  in  comparison  to  the  annual  investment  in  weapon  system 
modernization. 

There  could  well  be  cases  where  simulation  is  found  to  be  ineffective  or  too  costly 
for  training  or  both.  In  such  cases,  simulation  should  not  be  used  for  training.  This  could 
happen  for  a  variety  of  understandable  reasons.  Those  due  to  optimistic  expectations  of 
technological  feasibility  or  cost  are  obvious.  A  less  obvious  reason  is  that  reasonable 
training  plans  and  effective  integration  of  simulators  into  training  programs  in  ways  that  are 
acceptable  to  a  unit  and  its  command  are  required  to  assure  the  proper  and  successful  use  of 
a  simulator.  Information  presented  in  Chapter  FV  clearly  shows  that  user  acceptance  and 
the  existence  of  training  plans  are  central  to  the  successful  use  of  simulators;  simulators  that 
are  reported  to  have  problems  often  lack  training  plans.  Most  other  reported  limitations  of 
simulators  are  attributed  to  their  not  having  features  now  considered  desirable  by  users. 
Most  of  these  are  the  result,  variously,  of  outmoded  equipment,  desired  upgrades,  or 
insufficient  quantities  of  equipment.  Few  of  these  problems  present  requirements  for  new 
technology;  rather,  they  are  the  result  of  policy  judgments  made  not  to  provide  funds 
needed  for  upgraded  or  replacement  equipment 

One  may  question  the  rationale  that  says  simulators  should  be  acquired  if  they  train 
as  well  as  actual  equipment  but  cost  less  to  own  or  use.  This  guideline  is  used  to  justify  the 
current  acquisition  of  simulators  and  we  believe  that  it  should  be  reconsidered.  Remember 
that  new  weapon  systems  are  acquired  in  order  to  provide  performance  superior  to  that  of 
current  equipment,  in  order  quickly  to  overcome  a  potential  enemy,  and  to  minimize 
friendly  casualties.  Applying  the  same  rule  to  the  acquisition  of  new  training  equipment 
should  lead  us  to  prefer  increased  training  effectiveness  at  the  same  (or  less)  cost  than  that 
of  current  equipment  rather  than  equal  effectiveness  at  less  cost.  Increased  effectiveness  of 
training  equipment  at  increased  cost  may  also  make  sense,  as  it  does  for  weapons,  provided 
that  the  increase  in  effectiveness  has  military  value. 

There  are  several  ways  to  increase  the  effectiveness  of  a  training  simulator:  devise 
training  lessons  more  relevant  to  combat  requirements,  spend  more  time  in  training, 
develop  prompt  and  accurate  feedback  to  help  students  identify  error-producing  behavior, 
and  reinforce  correct  responses;  better  hardware  may  also  help.  All  simulators  incorporate 
computers,  and  adding  capabilities  that  enhance  learning,  such  as  those  noted  here,  is  well 
within  the  state-of-the-art.  Given  the  capability  of  computer-based  equipment  to  provide 
learning  opportunities  superior  to  those  available  in  field  training  (e.g.,  critical  events  in 
combat  scenarios,  objective  measurement,  quick  feedback,  accurate  diagnosis  of 
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performance,  and  informed  suggestions  for  improvement)  there  is  every  reason  to  believe 
that  advanced  distributed  simulation  can  provide  highly  effective  unit  training.  It  cannot 
replace  field  exercises.  The  development  of  training  strategies  to  improve  the  utility  of 
distributed  interactive  simulation  is  a  proper  and  realistic  goal  for  research  and  development 
on  military  training. 

Two  types  of  efforts  are  envisaged;  research  on  and  development  of  training 
strategies  for  DIS,  and  specific  application  and  tests  of  these  strategies  in  CCTT,  STOW, 
MDT2,  and  National  Guard  Training,  that  is,  the  new  DIS  systems  now  under 
development.  Though  these  systems  share  many  common  technical  features,  they  differ 
both  in  purpose  of  use  and,  as  a  consequence,  in  details  of  design.  It  is  important  to  note 
that  the  major  emphasis  in  R&D  on  methods  of  training  is  still  given  to  individual  training, 
and,  of  that,  largely  to  training  in  institutions  and  not  to  on-the-job  training,  i.e.,  training  in 
units.  If  research  and  development  on  collective  training  has  not  been  neglected,  it  has 
surely  received  insufficient  attention.  Training  for  combat  readiness  places  a  priority  on 
collective,  rather  than  on  individual,  training. 

The  development  of  Distributed  Interactive  Simulation  (also  called  Advanced 
Distributed  Simulation)  systems  that  will  be  used  for  Service  and  joint  training  is  so  heavily 
dependent  on  new  technology  in  such  areas  as  networking,  semi-automated  forces,  and 
dynamic  terrain  and  environment  that  consideration  of  how  best  to  use  DIS  for  training  has 
fallen  behind.  Service  experience  with  simulators  (reported  in  Chapter  IV)  clearly  shows 
that  inadequate  or  non-existent  training  plans  have  seriously  impaired  the  utility  of  current 
simulations  and  some  future  history  may  include  DIS  unless  we  act  soon  to  develop 
training  plans  for  their  use.  Note  that  twice  as  many  funds  are  being  spent  now  for  R«&;D 
on  training  equipment  than  on  training  procedures;  this  is  also  the  pattern  for  the  Multi- 
Service  Distributed  Training  Testbed  (MDT2)  and  perhaps  (we  speculate)  for  CCTT  and 
STOW.  In  1984,  the  Air  Force  issued  guidance  to  "develop  system  training  plans  for 
simulators"  (Simulation  Broad  Area  Review,  1984);  in  1993,  the  Air  Force  Four-Star 
Flight  Simulator  Review  found  that  "most  weapon  systems  do  not  have  system  training 
plans." 

Well  before  SIMNET  was  started.  Col  Jack  Thorpe,  USAF  (Ret.)  who  led  the 
development  of  SIMNET,  wrote  in  a  paper  on  visual  systems  for  flight  simulators: 

There  is  evidence  to  suggest  that  how  a  training  device  is  used  often 
accounts  for  more  training  output  (efficiency,  as  well  as  effectiveness)  than 
the  hardware  characteristics  of  the  device.  Sophisticated  hardware 
refinements  of  a  device  might  result  in  only  token  increases  in  training 


effectiveness,  whereas  simple  innovations  in  the  use  of  the  device  might 

substantially  increase  its  effectiveness.  (Thorpe,  Varney  et  al. ,  1978) 

This  is  also  the  major  theme  developed  by  Lt.  Gen.  Frederic  J.  Brown,  USA  (Retired), 
with  his  emphasis  on  the  need  for  training  development  to  structure  "...  the  training  process 
to  ensure  that  specific  training  events  occur  in  the  manner  and  sequence  desired  to  achieve 
intended  task  training  purposes"  (Brown,  1993).  Research  and  development  on  training 
strategies  has,  up  to  now,  been  focused  on  individual  training,  with  much  less  attention 
given  to  crew  and  unit  training.  This  is  hard  to  understand  or  to  explain  since  effectiveness 
in  combat  depends  on  crew  and  unit  training  for  which  individual  training  is  a  necessary 
but  insufficient  condition  for  success.  R&D  on  unit  training  and  on  performance 
measurement  of  units  in  combat  exercises  and  in  command  centers  must  clearly  be  given  a 
much  higher  priority  than  it  has  at  present.  The  issue  of  performance  measurement  of  joint 
readiness,  the  product  of  training,  to  supplement  commander's  estimates  of  readiness,  is 
now  being  addressed  by  The  Joint  Staff. 

R&D  on  unit  training  will  not  be  enough  to  ensure  warfighting  effectiveness  since 
each  Service  is  responsible  for  its  own  component  training  and  not  for  Joint  training. 
However,  actual  combat  occurs  as  a  joint,  rather  than  exclusively  a  Service,  activity.  Some 
Service  R&D  now  underway  could  contribute  significantly  to  joint  training;  this  includes 
Army  work  on  Battle  Command  Staff  Training,  the  Unit  Performance  Measurement 
System  and  analyses  of  exercises  at  the  National  Training  Center,  Navy  work  on  Tactical 
Decision  Making  under  Stress,  and  Air  Force  efforts  on  MultiRAD  program. 

The  Multi- Service  Distributed  Training  Testbed  (MDT2)  program  is  the  only  multi¬ 
service  R&D  program  known  to  us  that  considers  joint  training;  STOW  may  do  so  in  the 
future,  MDT2  is  a  testbed  that  is  being  built  and  whose  functions  must  be  tested  to  see  if 
they  will  work  as  intended;  it  will  use  a  close  air  support  scenario  for  the  initial 
demonstration  of  its  feasibility.  The  use  of  MDT2  for  research  and  development  on 
training  technology  for  joint  combat  will  not  start  before  FY  1995,  at  a  level  estimated  to  be 
about  $2  million  per  year.  Given  the  importance  of  joint  training  for  joint  readiness,  this 
multi-Service  program  should  be  strongly  supported  and  enhanced  almost  immediately  so 
that  serious  attention  can  be  given  to  development  and  test  of  joint  training  strategies  for 
such  missions  as  joint  air  defense,  joint  amphibious  and  land  assault,  and  joint  close  air 
support  (which  will  be  demonstrated  but  not  developed  in  the  current  effort).  Work  with 
MDT2  on  joint  training  readiness  should  obviously  be  coordinated  with  the  Army's  CCTT 
and  AVCATT  programs  and  with  ARPA's  efforts  on  STOW  and  National  Guard  training. 
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DMSO,  ARP  A,  and  DMDC/TREAD,  rather  than  the  Services,  appear  to  be  the  only  OSD 
agencies  that,  with  support  by  CJCS  and  the  CINCs,  are  in  a  position  to  sponsor  and 
support  R&D  on  training  strategies  for  joint  combat  missions. 

Under  the  best  of  circumstances,  it  will  be  difficult  to  compare  the  effectiveness  of 
DIS  systems  for  training  with  that  of  actual  equipment,  i.e.,  in  field  exercises  at  Service 
and  Joint  levels  on  instrumented  ranges.  Here,  we  are  again  concerned  with  how  best  to 
evaluate  the  effectiveness  of  DIS  training  with,  e.g.,  BFTT,  TCTS,  JACTS,  MDT2, 
CCTT,  STOW,  and  the  National  Guard  system.  All  of  these  systems  are  intended  to 
support  combat  training  exercises  and  a  key  problem  will  be  the  availability  of  troops, 
commanders,  and  the  personnel  needed  to  conduct  tests  to  determine  training  effectiveness 
and,  later,  to  estimate  joint  training  readiness.  The  need  to  engage  relatively  large  forces 
both  in  DIS  and  in  live  exercises,  in  order  to  compare  the  effectiveness  of  each,  means  that 
such  opportunities  will  be  few  and  perhaps  not  sufficient  to  provide  credible  test  data. 
Advantage  should  be  taken  of  the  opportunity  to  collect  training  effectiveness  data  in  many 
scheduled  exercises  on  a  not-to-interfere  basis  for  use  in  a  joint  training  performance  data 
base,  perhaps  as  an  extension  to  the  Joint  Universal  Lessons  Learned  System  (JULLS).  At 
present,  JULLS  provides  descriptive  narrative  information  about  exercises  rather  than 
objective  performance  measurement  data. 

This  means,  necessarily,  that  attention  must  also  be  given  to  the  planning  and 
conduct  of  tests  meant  to  evaluate  the  effectiveness  of  training  in  DIS  systems.  There  is  a 
need  to  collect  performance  data  on  unit  training  wherever  it  may  occur  and  to  develop  a 
datahasp.  that  could  yield  reliable  data  on  unit  training  from  independent  events  that  were  not 
planned  to  be  part  of  a  single,  systematic,  large-scale  experiment.  Consideration  should  be 
given  soon  to  this  possibility.  Attention  should  also  be  given  to  holding  a  workshop  and 
establishing  special  interest  groups  to  encourage  the  exchange  of  information  and 
coordination  of  activities  on  such  topics  as  training  strategies  for  unit  and  joint  training, 
data  bases  to  support  meta-analyses  of  unit  training,  and  design  of  DIS  systems  to  improve 
the  effectiveness  of  these  simulations  for  training. 

Discrepancies  observed  with  currently  used  simulations  are  largely  the  result  of 
aging,  i.e.,  in  contrast  to  what  current  technology  advances  could  now  make  available  and 
of  constraints  when  they  were  procured  or  of  decisions  not  to  upgrade  them  now.  A  major 
limitation  in  some  simulations  is  the  absence  of  adequate  training  plans,  despite  guidelines 
that  such  plans  are  needed  to  insure  proper  use  of  a  simulation  and  its  integration  in  the 
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overall  training  plan.  This  is  a  management  rather  than  a  technological  issue  and  has 
existed  for  the  past  15  to  20  years. 

Although  this  paper  is  directed  primarily  towards  the  capabilities  and  effectiveness 
of  simulation  technology  for  training,  it  is  inappropriate  to  disregard  the  issue  of  cost. 
Money  is  never  an  unlimited  asset  and  resources  allocated  to  one  type  of  training  or  training 
technology  implies  that  fewer  resources  can  be  allocated  to  some  other  aspect  of  training 
technology.  In  the  real  world,  policy  with  respect  to  how  to  train  (e.g.,  in  schools  or  on- 
the-job)  or  what  type  of  R&D  to  promote  (e.g.,  individual  or  collective  training),  or  what 
type  of  training  equipment  to  develop  and  procure  is  expressed  ultimately  as  a  series  of 
budget  decisions.  Thus,  it  is  naive  to  believe  that  information  about  training  methods  and 
equipment  or  about  their  effectiveness  can  be  useful  without  parallel  information  about  the 
cost  of  acquiring  and  using  these  methods  of  training.  Therefore,  it  seems  necessary  to 
observe  that  only  limited  and  insufficient  attention  has  been  given  to  the  cost  and 
effectiveness  of  various  types  of  simulation  relevant  to  training  and  that  meaningful  policy 
on  R&D,  acquisition,  and  training  itself  is  thereby  constrained.  Prime  areas  that  need  better 
cost  data,  as  well  as  effectiveness  data,  are  those  that  clearly  have  high  costs:  these  appear 
to  be  aviation  training  and  training  using  distributed  interactive  simulation,  because  of  its 
significance  for  joint  readiness;  the  proper  mix  of  school  and  on-the-job  training  is  another 
key  area. 

There  is  nothing  inherently  difficult  about  collecting  cost  data  on  matters  related  to 
simulation  and  training,  except  perhaps  generating  a  desire  to  do  so.  Such  an  effort  could 
proceed  in  two  phases:  (1)  defining  the  relevant  cost  elements  of  interest  and  (2)  examining 
the  appropriation  accounts  that  provide  the  specified  funds.  In  the  case  of  individual 
training,  for  example,  the  cost  elements  of  interest  include  pay  and  allowances  of  students 
while  they  are  being  trained,  pay  and  allowances  of  military  and  civilian  personnel  who 
provide  and  support  training,  base  operating  costs  of  the  training  establishment,  investment 
costs  for  instruction  and  procurement  related  to  training,  and  the  overhead  costs  for 
administration  and  command  of  the  training  agency  (MMTR  FY  1995, 1994,  p.  80).  The 
amounts  of  funds  allocated  to  these  cost  elements  are  found  in  the  following  Appropriation 
Accounts:  Operation  and  Maintenance,  Military  Personnel,  Reserve  Personnel,  National 
Guard  Personnel,  Aircraft  Procurement,  Missile  Procurement,  Weapons  Procurement, 
Other  Procurement,  Military  Construction,  and  Research  and  Development.  The  Services 
use  various  combinations  of  these  appropriations  to  support  individual  training  (MMTR  FY 
1995, 1994,  Appendix  D). 
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Obviously,  other  cost  elements  and  appropriation  accounts  apply  to  OPTEMPO, 
collective  training,  procurement,  and  research  and  development  related  to  training.  Except 
for  individual  training  in  institutions,  there  is  no  regular  compilation  of  data  on  the  costs  of 
individual  training  in  units,  collective  training  in  institutions  or  in  units,  OPTEMPO,  or  of 
various  types  of  simulation  for  training.  The  regular  reporting  of  training-related  cost  data 
makes  it  possible  to  identify  trends  that  may  have  significance  for  readiness  while 
contributing,  at  the  same  time,  to  the  correction  and  refinement  of  the  data  that  are  being 
reported. 

Skepticism  about  the  value  of  simulators  for  training — ^regardless  of  their  actual 
merit— is  an  understandable  response  to  the  threat  that  simulators  pose  to  OPTEMPO.  A 
standard  remark  is  that  "no  pilot  joined  the  Services  so  that  he  could  fly  in  a  box." 
Simulators  pose  a  real  threat  to  OPTEMPO  because  the  cost  of  buying  and  using  simulators 
can  be  balanced  by  a  reduction  in  funds  for  flying  hours,  vehicle  miles,  or  steaming  hours 
per  year.  If  not  universal,  resistance  to  the  use  of  simulators  is  most  pronounced  among 
aviators  of  all  Services,  led  by  fighter  pilots,  with  middling  acceptance  of  simulators  by 
transport  pilots.  There  appears  to  be  little  resistance  to  the  use  of  simulators  for  training 
armor  forces  in  the  Army,  where  the  cost  of  the  Close  Combat  Tactical  Trainer  (about  $2 
billion)  will  be  offset  by  a  reduction  in  funds  for  vehicle  miles  over  the  life  of  the  CCTT 
(Noble  and  Johnson,  1991).  Command  post  exercises  are  an  accepted  method  of  training 
at  all  levels  in  all  Services:  component,  multi-Service,  joint,  and  coalition.  Such  exercises 
take  place  ("on-the-job")  in  actual  command  centers  in  units,  as  well  as  in  gaming  centers  in 
institutions;  only  the  combat  models  used  in  such  exercises  may  be  challenged  (but  not 
often  enough:  see  Davis  and  Blumenthal,  1991).  Overall,  we  may  observe  that  simulators 
and  simulations  are  tools  for  training  and  for  assessing  readiness.  They  have  various 
strengths  and  limitations  for  various  types  of  applications.  Assuring  that  the  significant 
limitations  of  simulation  and  modeling  are  understood,  the  significant  issue  is  to  determine 
the  optimum  combinations  of  simulation  and  actual  equipment  that  can  provide  the  most 
effective  training  at  the  least  cost,  as  appropriate  for  various  types  of  training,  up  to  the 
joint  level.  That  would  be  a  worthy  goal. 

The  paradigm  that  identifies  "who  is  trained"  and  "where  training  occurs"  is  a 
useful  and  easily  understood  tool  for  assessing  what  is  known  about  the  effectiveness  and 
cost  of  all  types  of  military  training  in  the  environments  in  which  training  takes  place. 
Training  occurs  throughout  a  persons'  military  career,  and  in  various  places  and 
environments.  This  perspective  has  helped  us  note  that  less  is  known  than  is  desirable. 
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about  the  cost  and  effectiveness  of  on-the-job  training  (i.e.,  training  that  occurs  in  a  non¬ 
training  environment)  or  about  the  cost  of  collective  training,  in  institutions  as  well  as  in 
operational  units.  We  believe  that  the  same  paradigm  has  utility  for  developing  budget  data 
useful  for  developing  policy  on  expenditures  for  RDT&E,  procurement  and  utilization 
(operation  and  maintenance)  of  training  at  individual,  unit,  and  joint  levels. 


VIII.  FINDINGS  AND  RECOMMENDATIONS 


This  paper  considers  issues  with  respect  to  research  and  development,  procurement 
and  utilization  that  are  central  to  the  development  of  policy  concerning  the  use  of  simulation 
for  military  training.  Findings  and  Recommendations  focus  on  budgets  related  to 
simulation  and  training,  the  cost  and  effectiveness  of  simulation,  experience  of  the  Services 
with  simulation,  distributed  interactive  simulation,  and  the  technologies  relevant  to 
simulation  and  training. 

FINDINGS 

Budgets  Related  to  Simulation  and  Training 

Budget  data  on  the  costs  of  simulation  and  training  are  not  reported  regularly  in  the 
Department  of  Defense.  This  means  that  our  ability  to  discern  trends  in  costs  of  different 
types  of  training  (e.g.,  individual  or  collective  training),  or  to  compare  the  costs  of  training 
at  a  school  to  training  in  an  operational  unit,  or  to  direct  research  and  development  towards 
areas  of  highest  pay-off  for  training  effectiveness  and  cost  is  seriously  limited.  The  costs 
of  OPTEMPO  are  not  well  known;  in  fact,  we  found  two  different  estimates  of  the  costs  of 
OPTEMPO  that  differ  by  a  factor  of  2.3.  In  this  report,  data  from  various  sources  are 
compiled  to  estimate  what  some  of  these  costs  are;  some  of  the  data  are  highly  reliable, 
while  some  lack  clear  definition  as  to  what  cost  elements  may  or  may  not  be  included.  Our 
findings  are  summarized  in  Figure  VDI-l. 

Individual  training  occurs  at  schools  and  cost  $14.4  billion  in  FY  94.  Collective 
training  occurs  in  operational  units  but  its  costs  are  not  reported  regularly;  it  was  estimated 
to  cost  $12.7  billion  in  FY  93.  Two  estimates  were  found  for  expenditures  for  OPTEMPO 
(fuel,  consumables,  repairs  and  maintenance  for  flying  hours,  steaming  days,  and  vehicle 
miles):  $9.4  billion  (FY  1993,  LMI)  and  $21.4  billion  (FY  1991,  Angier,  Alluisi  and 
Horowitz).  The  larger  estimate  includes  costs  for  repair  and  depot  maintenance  not 
included  in  the  smaller  estimate;  except  for  this  item,  we  were  not  able  to  resolve  the  basis 
for  these  widely  different  estimates.  The  fact  that  two  FFRDCs  developed  such  different 
estimates  for  the  cost  of  OPTEMPO  illustrates  the  importance  of  having  reliable  and  regular 
estimates  of  the  costs  associated  with  various  aspects  of  simulation  and  training. 
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Type  of  Expenditure 


RPT&E 

Simulators  for  weapon  systems 
Technology 

Training  equipment 
Training  methods 
Modeling  and  simulation 
Joint  commands 
DMSO 
ARPA 


Procurement  of  simulators 
Aviation  $  0.800 

Non-system  devices  0.243 


0.057 


Amount^ 

Period 

Source 

$0,336 

FY91-97 

Frost  &  Sullivan  (1 993)2 

0.101 

FY94 

MATRIS  (1993)3 

0.038 

FY94 

MATRIS  (1993)3 

0.019 

FY  91-92 

IG  (1993)4 

0.073 

FY  93-94 

DMSO  (1994)5 

0.103 

FY  92-97 

ARPA  (1993)6 

FY  91-97 

Frost  &  Sullivan  (1993) 

1.100 

NA 

NA 

14.4 

FY94 

MMTR  (1993)7 

NA 

NA 

12.7 

FY93 

LMI  (1993)8 

9.4 

FY93 

LMI  (1993) 

21.4 

FY91 

Angieret  al.  (1992)8 

0.425 

FY94 

Briefing  material 

0.369 

FY  91-97 

Frost  &  Sullivan  (1 993) 

Alt  others 

Models  and  simulations 
Military  Constmction 


Individual  training  in  institutions 
Individual  training  in  units 
Collective  training  in  institutions 
Collective  training  in  units 
OPTEMPO 

Joint  exercises 
Simulator  maintenance 


■'  Amount,  in  billions,  for  year  crted  or  average  of  years  noted. 

2  Frost  &  Sullivan  (1993). 

^  Manpower  and  Training  Research  Information  System  (1993). 

*  Inspector  General  (1993). 

^  Defense  Modeling  and  Simulation  Office  (1994). 

®  Advanced  Research  Projects  Agency  (1 993). 

^  Military  Manpower  Training  Report,  FY  94  (1993). 

®  Logistics  Management  InstKute  (1993). 

®  Angler,  Alluisi,  and  Horowitz  (1 992). 

NA;  Not  Available 

Figure  VIIM.  Estimated  Annual  Expenditures  for  Training  and  Simulation,  in 
Terms  of  RDT&E,  Initial  Investment,  and  Operating  and  Support  of  Training 
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The  procurement  of  simulators  for  training  costs  about  $1.1  billion  per  year 
(average  of  FY  91-97);  RDT&E  and  support  of  simulators  cost  an  additional  $0.5  and  $0.4 
billion,  respectively.  The  most  expensive  simulators  are  for  aviation;  they  cost  about  $0.8 
billion  per  year  to  procure  (73  percent  of  the  costs  of  simulators  for  all  types  of  weapon 
systems);  non-system  training  devices  cost  $0.2  billion  (22  percent);  simulators  for  all 
other  types  of  weapons  cost  $0.6  billion  per  year  (5  percent). 

Expenditures  for  RDT&E  on  simulators  and  training  equipment  average  about  $0.4 
billion  per  year,  training  methods  about  $0.04  billion,  and  modeling  and  simulation  (for 
DMSO,  ARPA,  and  the  Joint  commands)  about  $0. 19  billion  per  year. 

A  significant  issue  in  the  cost  of  training  concerns  who  is  trained  and  where  training 
occurs.  "Who  is  trained"  refers  to  individual  and  collective  training;  "Where  training 
occurs"  refers  to  training  at  institutions  (i.e.,  schools)  or  in  the  operational  units.  The 
problem  in  ascertaining  the  cost  of  each  type  of  training  at  various  places  (and  the  related 
issue  of  evaluating  their  cost  and  effectiveness)  arises  because,  except  for  individual 
training  at  institutions,  there  is  no  regular  or  consistent  report  that  identifies  the  cost  of  all 
other  types  of  training.  Our  estimates  of  what  these  costs  are  were  compiled  from  a  variety 
of  sources,  as  shown  in  Figure  Vin-2.  The  costs  of  RDT&E  and  initial  investment  are  not 
included  in  this  figure;  we  were  not  able  to  find  any  usable  data  on  the  costs  of  collective 
training  in  institutions  or  of  individual  training  in  units.  We  estimate  individual  training  in 
institutions  to  cost  $14.4  billion  in  FY  94;  this  is  a  reliable  figure.  Other  costs,  that  are 
estimates  by  a  variety  of  sources,  are:  collective  training  in  units  cost  $12.7  billion  in  FY 
1993,  OPTEMPO  cost  either  $9.4  or  $21.4  billion  (FY  91),  according  to  two  different 
estimates;  and  joint  exercises  cost  about  $0,425  billion  (FY  1994). 

Providing  reliable,  consistent,  and  regularly  updated  data  on  the  actual  costs  of 
individual  and  collective  training  at  institutions  and  units  is  a  matter  of  the  utmost  urgency, 
since  these  expenditures  directly  influence  the  readiness  of  our  troops. 

Cost  and  Enectiveness  of  Simulation 

Evaluations  of  the  cost  and  effectiveness  of  simulation  for  training  are  based  almost 
exclusively  on  the  use  of  flight  and  maintenance  simulators  and  of  computer-based 
instruction  for  initial,  individual  training  at  institutions.  The  cost  and  effectiveness  of 
simulation  for  more  advanced  individual  training  in  units  or  for  collective  training  at 
institutions  or  in  units  for  component  or  joint  training  has  not  received  much  attention. 


Where  Training  Occurs 


Institution 

Unit 

Who  is 

trained 

Bv  service.  FY  94’ 

Army 

Navy 

Marine  Corps 

Air  Force 

Bvtvpe,  FY  94'' 

$5.4  B 
4.5 

1.3 

14.4 

Not  available: 

Individual  (on-the-job)  training  in 
units  for  those  trained  only  on  the 
job,  job  familiarization  for  those 

Individual 

Recruit  $1 .2  B 

Officer  acquisition  .5 

Specialized  skill  4.1 

Flight  22 

Professional  development  .9 

One  station  (Armv)  .3 

14.4 

trained  at  schools,  and  training  for 
those  assigned  jobs  that  do  not 
match  their  MOS® 

Unit  training  $12.7  FY  93^ 

Collective 

Army  Combat  Training 
Centers,  FY  95 

.5  B® 

OPTEMPO  9.4  FY  93^ 

21.4  FY913 

Other  data  not  available 

Joint  exercises  0.425  FY  94^ 

1MMTR  (1993). 

2LMI  (1993). 

3  Angier,  Alluisi,  and  Horowitz  (1992). 

*  Briefing  Estirnate. 

5  AUSA  (1994):  about  one-third  of  this  total  is  for  transport  to  and  from  the  training  center  (Fig.  11-16). 

6  CBO  (1994)  estimates  6-10  percent  mismatch  between  job  and  MOS  in  the  Army,  FY  1993  (p.  32). 

Figure  VIII-2.  Estimated  Annual  Expenditures  for  Training,  by  Type  of  Training, 
and  Where  Training  Occurs  (does  not  include  RDT&E  and  initial  investment) 

The  available  findings  show  that  simulators  are  cost-effective  for  initial  flight  and 
maintenance  training  in  institutions:  they  train  as  well  as  does  actual  equipment  and  cost 
less  to  procure  and  use.  This  finding  applies  also  to  computer-based  instruction,  in 
comparison  to  conventional  classroom  instruction.  Simulators  are  a  good  investment.  The 
cost  of  their  procurement  can  be  amortized  in  periods  of  one  to  four  years.  However, 
optimum  combinations  for  the  use  of  simulators  and  actual  equipment  for  various  types  of 


training  have  not  been  studied;  nor  have  such  critical  issues  as  the  rates  of  learning  and 
forgetting,  which  are  basic  to  determining  how  much  and  when  simulators  or  actual 
equipment  are  best  used  for  initial  and  refresher  training.  The  decision  to  use  new 
simulators  on  the  basis  of  equal  effectiveness  and  less  cost  than  actual  equipment,  the  rule 
used  at  present,  overlooks  the  fact  that,  for  military  purposes,  one  should  seek  simulators 
that  provide  increased  performance  effectiveness  at  the  same  or  lesser  cost. 

Service  Experience  with  Simulation 

Information  was  collected  on  over  50  simulations  considered  by  the  four  military 
services  to  be  successful  or  to  have  problems.  Features  associated  with  successful 
simulations  are  high-user  acceptance,  timely  availability  of  a  well-developed  training  plan  to 
show  how  the  simulator  should  be  used,  genuine  contribution  of  the  simulator  to  training 
demonstrated  by  features  for  performance  measurement  and  feedback,  acceptable  costs, 
and  minimum  interference  by  simulators  with  existing  norms  for  training  with  actual 
equipment,  i.e.,  little  reduction  in  budgets  for  flying  hours,  vehicle  miles,  and  steaming 
hours.  Problems  with  existing  simulations  are  attributed  to  inadequate  or  non-existent 
training  plans,  discrepancies  between  the  performance  of  simulators  and  actual  equipment, 
and  the  absence  of  features  considered  to  be  important  for  training,  e.g.,  motion  platforms, 
sensors  (IR,  radar,  EW),  feedback  capability.  Almost  all  of  these  limitations  are  the  result 
of  decisions,  for  cost  or  other  reasons,  to  procure  simulations  with  limited  or  no  capability 
for  training  on  certain  tasks.  Current  technology  appears  adequate  to  deal  with  most  of  the 
deficiencies  that  were  reported,  provided  that  there  is  an  interest  in  and  funds  are 
appropriated  to  upgrade  and  improve  current  equipment 

The  following  trends  are  observable  with  respect  to  the  role  of  simulators  for 
training:  (1)  a  reduction  of  flying  hours  for  training,  up  to  50  percent  per  year  for 
transport  aircraft  and  of  vehicle  miles  per  year  for  armor,  to  pay  for  the  cost  of  procuring  a 
new  distributed  training  system  for  close  combat;  (2)  use  of  simulators  to  complement 
flight  training  for  advanced  combat  aircraft,  with  a  slight  reduction  in  flying  hours;  and  (3) 
the  development  of  low-cost,  unit-training  devices  to  be  placed  at  most  flight  bases  for  use 
as  modifiable,  part-task  trainer  for,  e.g.,  electronic  warfare,  target  recognition,  and  digital 
terrain  land  mass  training.  Current  flight  simulators  will  be  phased  out  because  they  are  too 
expensive  to  be  placed  at  all  flight  bases.  A  significant  trend  is  the  development  and  use  of 
advanced  distributed  simulators  in  all  Services  for  collective,  combat  training  but  it  is  too 
soon  to  assess  their  utility  for  training. 


Distributed  Interactive  Simulation 

Distributed  Interactive  Simulation  (DIS)  systems  now  being  developed  have  a  great 
potential  for  improving  unit  and  joint  training  and  joint  readiness  (and  will  also  have 
important  applications  in  other  areas,  such  as  test  and  evaluation,  tactics  and  doctrine,  and 
mission  rehearsal).  However,  the  development  of  training  strategies,  i.e.,  ways  to  use  DIS 
systems  to  improve  collective  training,  is  lagging  behind  development  of  the  underlying 
technology  in  hardware  and  software,  communications  and  standards  needed  to  support 
DIS.  A  problem  that  will  surely  arise  is  how  best  to  use  these  systems  for  training  (and  test 
and  evaluation)  and  how  to  evaluate  their  effectiveness  and  cost.  Meaningful  tests  will 
require  large  numbers  of  people  to  serve  as  test  subjects  over  extended  periods  of  time,  as 
well  as  scenarios  and  performance  measuring  techniques  that  must  still  be  developed  and 
tested  in  order  to  be  available  when  needed;  this  also  applies  to  the  training  of  personnel  to 
design  and  conduct  tests  and  collect  reliable  test  data  needed  to  evaluate  the  utility  of  DIS 
for  training  and  to  support  a  decision  to  procure  the  required  equipment 

Technologies  Relevant  to  Simulation  and  Training 

A  review  of  technologies  critical  to  simulation  and  training  included  R&D  on 
distributed  interactive  simulation  in  areas  concerned  with  networks,  semi-automated  forces, 
terrain  and  environment,  range  instrumentation,  individual  combatants,  virtual  environ¬ 
ments,  and  training  technology.  Areas  that  receive  major  funding  ($1(X)  million  a  year  or 
more)  are  networks,  terrain  and  environment,  and  range  instrumentation.  Within  these 
areas,  increased  attention  should  be  given  to  develop  the  Asynchronous  Transfer  Mode 
(ATM)  protocol  for  DIS,  support  for  exercise  management,  compatibility  between  data 
bases  for  terrain  and  environment,  radio-frequency  network  bandwidths  for  range 
instrumentation,  and  communication  standards.  Less  support  is  being  given  to  methods  of 
training,  design,  and  use  of  individual  combatants  in  DIS  and  SAPOR.  There  is  a  need  for 
methods  of  evaluating  performance  in  all  areas,  clarification  of  appropriate  representations 
of  individual  combatants  (including  dual-use  potential)  in  DIS,  and  effective  ways  to  use 
and  measure  the  training  potential  of  most  DIS  systems. 

RECOMMENDATIONS 

Priorities  for  Research  and  Development 

Review  the  research  and  development  programs  on  simulation  and  training  to 
assure  that  they  focus  on  areas  of  highest  expenditures  and  greatest  potential  payoff.  These 


areas  are  aviation  simulators,  aviation  training,  OPTEMPO,  joint  training  and  readiness, 
and  distributed  interactive  simulation. 

Distributed  Interactive  Simulation 

Extend  efforts  to  evaluate  the  cost  and  effectiveness  of  training  technology  beyond 
the  limited  areas  of  flight  and  maintenance  simulators  and  computer-based  instruction. 
Attention  should  be  given  to  methods  of  evaluating  the  cost  and  effectiveness  of  distributed 
interactive  simulation  systems,  such  as  the  Close  Combat  Tactical  Trainer  (CC  IT), 
Synthetic  Theater  of  War  (STOW),  and  the  Multi-Service  Training  Testbed  (MDT2),  and  to 
the  use  of  modeling  for  joint  training,  where  the  major  issues  are  likely  to  be  validity  and 
effectiveness. 

Training  Strategies 

Give  high  priority  to  the  development  and  evaluation  of  training  strategies  for  the 
use  of  distributed  interactive  simulation  for  large-scale  Service  training  and  joint  training  for 
readiness.  Emphasize  efforts  concerned  with  estimating  performance  in  Joint  training 
exercises  because  this  will  provide  a  way  to  estimate  joint  training  readiness.  Development 
and  evaluation  of  methods  of  measuring  performance  in  joint  training  should  be  started 
soon  in  order  to  be  in  place  when  needed  to  evaluate  the  new  DIS  systems,  as  they  become 
available  in  three  to  five  years. 

Performance  Data  Base 

Develop  a  data  base  system  that  provides  a  systematic  way  to  compile  performance 
data  that  become  available  from  large-scale  Service  and  joint  exercises.  Evaluating  the 
effectiveness  of  new  DIS  systems  for  training  (e.g.,  CATT,  MDT2,  and  STOW)  will  be 
difficult  because  of  the  large  resources  needed  for  test  subjects  (military  personnel  for 
extended  exercises),  data  collection,  and  personnel  qualified  to  conduct  tests  and  analyze 
results.  The  reason  to  develop  a  comprehensive  data  base  is  that  it  can  provide  a  means  to 
determine  lessons  learned  from  many  different  tests  to  supplement  the  large-scale  test 
programs  that  will  be  expensive  and  difficult  to  conduct 

How  Much  Training  Is  Enough 

Support  research  and  development  on  key  issues  of  how  much  training  is  enough 
and  how  often  refresher  training  must  occur  to  maintain  joint  training  readiness.  There  is  a 


significant  absence  of  critical  information  on  training  with  respect  to  learning  and  forgetting 
curves  and  on  developing  optimum  combinations  of  the  use  of  simulators  and  actual 
equipment  for  various  applications.  This  must  be  remedied  in  order  to  maximize  the 
benefits  available  from  investments  in  different  equipment  and  methods  of  training. 

Combat  Models 

Insufficient  attention  has  been  paid  to  the  use  and  effectiveness  of  models  used  for 
joint  and  large-scale  Service  training.  A  program  to  evaluate  the  verification,  validity,  and 
accreditation  of  models  used  in  unit,  battle  command  staff,  and  joint  training  should  be 
undertaken. 

Cost-Effectiveness  Paradigm 

The  utility  of  simulations  has  generally  been  decided  on  the  basis  of  equal 
effectiveness  and  less  cost  than  the  use  of  actual  equipment.  Although  this  is  an  acceptable 
guideline,  attention  should  be  directed  towards  the  development  of  simulations  that  increase 
performance  effectiveness  at  no  appreciable  increase  in  cost  or  at  lesser  cost.  Military 
effectiveness  benefits  from  training  for  improved  performance  rather  than  merely  to  equal 
current  levels  of  proficiency. 

Cost  Data  on  Training 

Except  for  data  furnished  to  Congress  on  individual  training  in  institutions,  there  is 
a  notable  absence  of  regularly  reported  data  on  the  costs  of  collective  training  in  institutions 
and  operational  units,  on  the  costs  of  OPTEMPO,  exercises,  on-the-job  training,  joint 
training,  and  on  the  acquisition  of  training-related  hardware  and  software.  Undertake  the 
development  of  cost-reporting  systems  that  will  identify,  define,  and  make  regularly 
available  cost  data  on  training  needed  to  support  policy  decisions  in  each  of  these  areas. 

On-the-Job  Training 

Only  limited  attention  has  been  given  to  the  cost  and  effectiveness  of  on-the-job 
training,  in  comparison  to  formal  training  in  institutions.  Review  current  R&D  activities  in 
order  to  plan  a  more  vigorous  R&D  effort  toward  on-the-job  training.  New  developments 
in  computer-based  instruction,  distance  learning,  and  portable,  miniaturized  electronic  job 
aids  make  this  an  attractive  area  to  improve  the  effectiveness  and  reduce  the  costs  (largely 
hidden)  of  on-the-job  training. 
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APPENDIX  A 


ABBREVIATIONS 


ABBREVIATIONS 


A/AMTS 

Avionics/Armament  Maintenance  Trainer  System 

AAMRL 

Armstrong  Aerospace  Medical  Research  Laboratory 

AARS 

After  Action  Review  System 

ACC 

Air  Combat  Command 

ACM 

Air  Combat  Maneuvering 

ACMI 

Air  Combat  Maneuvering  Instrumentation 

ADF 

Air  Defense  Fighter 

ADS 

Advanced  Distributed  Simulation 

AETC 

Air  Education  and  Training  Command 

AFRES 

Air  Force  Reserve 

AFSOF 

Air  Force  Special  Operations  Force 

AFSPACECOM 

Air  Force  Space  Command 

AFT 

Avionics  Familiarization  Trainer 

AGESn 

Air-Ground  Engagement  System  n 

AGPT 

Ausbildungsegerat  Gefechtssimulator  Panzer  Truppe 

AI 

Artificial  Intelbgence 

AIS 

Aircraft  Instrumentation  System 

ATTSt 

Army  Integrated  Thermal  Signature  Target 

ALSA 

Air  Land  Sea  Agency 

ANG 

Air  National  Guard 

ARPA 

Advanced  Research  Projects  Agency 

ARPTT 

Aerial  Refueling  Part  Task  Trainer 

ASC 

Aeronautical  Systems  Center 

AB 

Army  Science  Board 

ARTEP 

Army  Training  Evaluation  Program 

ASD  (FM&P) 

Assistant  Secretary  of  Defense  (Force  Management  and  Personnel) 

ATCCS 

Army  Tactical  Command  and  Control  System 

AID 

Aircrew  Training  Device 

ATF 

Advanced  Tactical  Fighter 

ATM 

Asynchronous  Transfer  Mode 

ATP 

Acceptance  Test  Plan 

ATS 

Aircrew  Training  System 

AWSIM 

Air  Warfare  Simulation 

BBS 

Brigade/Battalion  Battle  Simulation 

BCTP 

Battle  Command  Training  Program 

BFIT 

Battle  Force  In-Port  Training 

BFTT 

Battle  Force  Tactical  Training 

BOPTT 

Boom  Operator  Part  Task  Trainer 

C/SCSC 

Cost/Schedule  Control  Systems  Criteria 

CADIS 

Communication  Architecture  for  Distributed  Interactive  Simulation 

CAP 

Combat  Air  Forces 

CAI 

Computer-Aided  Instruction 

CAT 

Canadian  Armor  Trophy 

CATS 

Combined  Arms  Training  Strategy 

CATT 

Combined  Arms  Tactical  Trainer 

CBI 

Computer-Based  Instruction 

CBITS 

Computer-Based  Instruction  Training  System 

CBS 

Corps  Battle  Simulation 

CBT 

Computer-Based  Training 

CCTS 

Combat  Crew  Training  Squadron 

CCTT 

Close  Combat  Tactical  Trainer 

CDR 

Critical  Design  Review 

CLS 

Contractor  Logistics  Support 

CMI 

Computer  Managed  Instruction 

CMSS 

Counterdrug  Modeling  and  Simulation  System 

COFT 

Conduct  of  Fire  Trainer 

CPAF 

Cost  Plus  Award  Fee 

CPT 

Cockpit  Procedures  Trainer 

CS 

Combat  Support 

CSRL 

Common  Strategic  Rotary  Launcher 

CSS 

Combat  Service  Support 

CTC 

Combat  Training  Center 

CTS 

Control  and  Tracking  System 

DBS 

Doppler  Beam  Sharpening 

DBTS 

Data  Base  Transformation  System 

DBWS 

Data  Base  Sork  Station 

DDS 

Display  and  Debriefing  System 

DD250 

Material  Inspection  and  Receiving  Report 

DemA^al 

DemonstrationA^ alidation 

DFAD 

Digital  Feature  Analysis  Data 

DEN 

Defense  Information  Infrastructure  Network 

DIS 

Distributed  Interactive  Simulation 

DISA. 

Defense  Information  Systems  Agency 

DMA 

Defense  Mapping  Agency 

DMSO 

Defense  Modeling  and  Simulation  Office 

DoD 

Department  of  Defense 

DRFP 

Draft  Request  For  Proposals 

DRLMS 

Digital  Radar  Landmass  Simulation 

DSB 

Defense  Science  Board 

DIED 

Digital  Terrain  and  Elevation  Data 

EAF 

Egyptian  Air  Force 

EC 

Electronic  Combat 

ECM 

Electronic  Counter  Measure 

ECP 

Engineering  Change  Proposal 

EMD 

Engineering  and  Manufacturing  Development 

ENJJPT 

Euro-NATO  Joint  Jet  Pilot  Training 

ENWGS 

Enhanced  Naval  Wargaming  System 

EPAFs 

European  Participating  Air  Forces 

EPG 

European  Participating  Government 

EPTM 

ENJJPT  Procedures  Trainer  Modernization 

ESIG 

Evans  and  Sutherland  Image  Generator 

EW 

Electronic  Warfare 

EWO 

Electronic  Warfare  Officer 

FAA 

Federal  Aviation  Administration 

FCA 

Functional  Configuration  Audit 

FCSMT 

Flight  Control  System  Maintenance  Trainer 

FEA 

Front  End  Analysis 

FMS 

Foreign  Military  Sales 

FOC 

Full  Operational  Capability 

FOT&E 

Follow-On  Operational  Test  and  Evaluation 

FRS 

Reet  Readiness  Squadron 

FSC 

Field  Service  Center 

FSCATT 

Fire  Support  Combined  Arms  Tactical  Trainer 

FSD 

Full  Scale  Development 

GFE 

Government  Furnished  Equipment 

H/SI 

Hardware/Software  Integration 

I/SWG 

Industry/Service  Working  Group 

DDRLMS 

Improved  Digital  Radar  Landmass  Simulation 

lEWTD 

Improved  Electronic  Warfare  Training  Device 

IOC 

Initial  Operating  Capability 

lOS 

Instructor  Operator  Station 

IR 

Infrared 

ISD 

Instructional  Systems  Design 

1ST 

Integrated  Systems  Trainer 

IVIS 

InterVehicular  Information  System 

JSFTS 

Joint  STARS  Flight  Training  System 

JARM 

Jammer,  Artillery,  Radar,  Missile 

JCM 

Joint  Conflict  Model  (new  version  of  Janus) 

JCS 

Joint  Chiefs  of  Staff 

JPO 

Joint  Program  Office 

JSORD 

Joint  System  Operational  Requirements  Document 

JSTARS 

Joint  Surveillance  Target  Attack  Radar  System 

kps 

kilobytes  per  second 

LANTIRN 

Low  Altitude  Navigation  and  Targeting  Infrared  System  for  Night 

LBO 

Launch  Base  Operations 

LFOV 

Limited  Field  of  View 

LGT 

Landing  Gear  Trainer 

LMI 

Logistics  Management  Institute 

LPTT 

LANTIRN  Part  Task  Trainer 

MAC 

Military  Airlift  Command 

MAD 

Magnetic  Anamoly  Detector 

MDT2 

Multi-service  Distributed  Training  Testbed 

MLU 

Mid  Life  Upgrade 

MMIR 

Military  Manpower  Training  Report 

MOB 

Main  Operating  Base 

MPTS 

Manpower,  Personnel,  Training  and  Safety 

MRD 

Mission  Rehearsal  Devices 

MSIP 

Multi-Stage  Improvement  Program 

MST 

Maintenance  Skills  Tutor 

MT 

Mission  Trainer 

MID, 

Maintenance  Training  Device 

MIE 

Maintenance  Training  Equipment 

NDRLMS 

New  Digital  Radar  Landmass  Simulation 

NTC 

National  Training  Center 

NWC 

Naval  War  College 

OAS 

Offensive  Avionics  Stations 

OBT 

On-Board  Trainer 

(XU 

Operational  Capabilities  Upgrade 

OFP 

Operational  Flight  Program 

OFT 

Operational  Flight  Trainer 

OJT 

On-the-Job  Training 

O&M 

Operations  and  Maintenance 

OM&S 

Operations  Management  and  Support 

OMAR 

Objectives  and  Media  Analysis  Report 

OPTEMPO 

Operating  Tempo 

OSMT 

Offensive  Station  Mission  Trainer 

OSS 

Offensive  Station  Simulator 

P&A 

Price  and  Availability 

PE 

Program  Element 

PCA 

Physical  Configuration  Audit 

PDR 

Preliminary  Design  Review 

PDU 

Protocol  Data  Unit 

PCS 

Precision  Gunnery  Simulator 

PMRT 

Program  Management  Responsibility  Transfer 

POM 

Program  Objective  Memorandum 

P/S 

Primary/Secondary 

PTT 

Part  Task  Trainer 

PVI 

Pilot  Vehicle  Interface 

R&D 

Research  and  Development 

RAA 

Required  Assets  Available 

RDT&E 

Research,  Development,  Test  and  Evaluation 

REFORGER 

Return  of  Forces  to  Germany 

RESA 

Research,  Evaluation  and  Systems  Analysis  (Navy  Simulation) 

RFI 

Request  for  Information 

RFP 

Request  for  Proposal 

RFT 

Ready  for  Training 

RFU 

Ready  for  Use 

RI 

Range  Instrumentation 

RWR 

Radar  Warning  Receiver 

SAC 

Strategic  Air  Command 

SAP 

Secretary  of  the  Air  Force 

SAR 

Synthetic  Aperture  Radar 

SCU 

Software  Capabilities  Upgrade 

SDA 

Simulator  Development  Activities 

SDIP 

Simulator  Data  Integrity  Program 

SECT 

Simulator  for  Electronic  Combat  Training 

SET 

Systems  Familiarization  Trainer 

SIMNET 

Simulator  Networking 

SMTS 

Simulated  Maintenance  Trainer  System 

SNS 

Satellite  Navigation  Station 

SOF 

Safety  of  Flight 

SON 

Statement  of  Need 

SOS 

Special  Operations  Simulation 

SRR 

System  Requirements  Review 

SSC 

Software  Support  Center 

SSDB 

Standard  Specification  Data  Base 

STOW 

Synthetic  Theater  of  War 

SUNT 

Specialized  Undergraduate  Navigator  Training 

TAC 

Tactical  Air  Command 

TACTS 

Tactical  Air  Combat  Training  System 

TAF 

Tactical  Air  Force 

TASWIT 

Tactical  Advanced  Simulated  Warfare  Integrated  Trainer 

TBD 

To  Be  Determined 

TC 

To  Complete 

TCT 

Total  Contract  Training 

TCTS 

Tactical  Combat  Training  System 

TEC 

Training  Extension  Course 

TFS 

Trainer  Flight  Simulator 

TIS 

Tracking  Instrumentation  System 

TMS 

Training  Management  System 

TRA 

Training  Requirements  Analysis 

TRAR 

Training  Requirements  Analysis  Reports 

IREAD 

Training  and  Readiness  Evaluation  Analysis  Division 

TSAR 

Training  System  Analysis  Report 

TSBAR 

Training  System  Basis  Analysis  Report 

TSRA 

Training  Systems  Requirement  Analysis 

TSRR 

Training  System  Readiness  Review 

TSSC 

Training  System  Support  Center 

TTNRT 

Table-Top  Navigational  Rendezvous  Trainer 

TUAF 

Turkish  Air  Force 

TWGSS 

Tank  Weapons  Gunnery  Simulation  System 

UCOFT 

Unit  Conduct  of  Fire  Trainer 

UPT 

Undergraduate  Pilot  Training 

UQT 

Unit  Qualification  Training 

USAF 

United  States  Air  Force 

USAFA 

United  States  Air  Force  Academy 

USAFE 

United  States  Air  Forces  in  Europe 

UTD 

Unit  Training  Device 

UTSS 

Universal  Threat  System  for  Simulators 

VCASS 

Visually  Coupled  Airborne  Systems  Simulator 

VE 

Virtual  Environment 

WAN 

Wide  Area  Network 

WARSIM 

Warfighters’  Simulation 

WST 

Weapon  System  Trainer 
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SERVICE  RESEARCH  AND  DEVELOPMENT  PROJECTS  ON 
SIMULATION  AND  TRAINING,  FY  1992-1995 
(SOURCE:  MATRIS,  1993) 


Service  Expenditures  for  R&D  on  Simuiation  and  Training,  FY  1992-1995 

(Dollars  in  Millions) 


FY  92 

FY  93 

FY  94 

FY  95 

Total 

Army 

81.9 

65.6 

71.9 

70.7 

292.1 

Navy 

19.6 

18.7 

13.9 

16.6 

68.9 

Air  Force 

53.9 

54.8 

53.2 

59.9 

222.2 

Total 

155.4 

139.1 

139.0 

147.3 

580.9 

STRICOM  Simulation  Training  and  Intelligence  Command 
isrrSC  National  Training  Center  (now  Naval  Air  Warfare  Center-T raining  Systems  Division) 
TSSSPO  Training  Simulation  Systems  Special  Project  Office 
Source:  Matris,  1993 


Distribution  of  Funds  to  Training  Equipment  Development  and  Training  R&D 

Agencies,  FY  1993,  by  Service 


Training  Equipment 

Training  R&D 

Total 

Army 

STRICOM 

$51.5  M 

ARI 

$  14.1  M 

$  65.6  M 

Navy 

NTSC 

16.6 

NADC 

2.1 

18.7 

Air  Force 

TSSSPO 

32.6 

AL/HRA 

22.2 

54.8 

Total 

100.7 

38.4 

139.1 

STRICOM  Simulation  Training  and  Intelligence  Command 
NTSC  National  Training  Center  (now  Naval  Air  Warfare  Center-T  raining  Systems  Division) 
TSSSPO  Training  Simulation  Systems  Special  Project  Office 
Source:  Matris,  1993 
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Service  R&D 


Project 

Number" 


A230 


A790-ST 


A790-HF 


A791-ET 


A791-MP 


A794 


Projects  Dealing  with  Simulation  and  Training, 
FY  1992-1995,  by  Service 
(Dollars  in  Millions) 


D574 


Total 


FY  92 


3.439 


3.769 


2.807 


3.962 


3.869 


4.222 


FY  93  FY  94  FY  95  Total 


23.442 


81.916 


8.483  4.809 


4.273 


4.806 


4.992 


14.617 


65.567 


8.688 


0.459 


DB39 

2.838 

2.561 

4.423 

D241 

27.816 

16.952 

48.207 

D396 

5.752 

2.547 

5.292 

D275 

0 

6.336 

0 

RM33D40 

1.751 

1.845 

RM33T24 

0.956 

1.249 

RM33T25 

0.3 

2.645 

LI  773 

4.666 

5.28 

R1889 

0.5 

2.5 

W2124 

0.827 

2.119 

S1427 

10.646 

3.1 

Total 

19.646 

18.738 

71.878 


1.053 


1.185 


1.889 


5.791 


4.0 


0 


13.918 


10.59 


0 


0 


4.563 


46.041 


5.000 


0 


70.745 


1.065 


1.26 


2.645 


6.583 


5.0 


16.553 


21.282 


3.769 


2.807 


27.513 


9.134 


9.214 


14.385 


139.016 


18.591 


6.336 


38.059 


290.106 


5.714 


4.65 


7.479 


22.32 


12.0 


2.946 


13.746 


68.885 


“  See  list  p.  B-6  ff. 
Source:  MATRIS,  1993 
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Service  R&D  Projects  Dealing  with  Simulation  and  Training, 
FY  1992-1995,  by  Service  (Dollars  in  Millions) 
(continued) 


Project 

Number 

FY  92 

FY  93 

FY  94 

FY  95 

Total 

Air  Force® 

1123® 

7.213 

8.711 

8.207 

8.887 

33.018 

2743® 

7.896 

5.927 

8.172 

8.608 

34.265 

3257 

4.452 

4.517 

3.69 

3.8 

18.388 

2325 

7.088 

3.6 

3.9 

4.179 

18.838 

2769 

8.981 

8.525 

2.6 

2.3 

22.406 

2851 

4.285 

3.6 

2.6 

1.5 

11.985 

2901 

5.91 

3.7 

0 

0 

9.61 

3000 

0 

2 

1.5 

3.5 

2968 

0.4 

0.62 

0 

0 

1.02 

3282 

0.67 

1.1 

0.7 

0 

2.47 

3772 

3.561 

0.805 

0.6 

0 

4.966 

4022 

2.589 

10.674 

8.762 

1.7 

23.725 

4033 

0 

0 

7.142 

22.707 

29.849 

3818 

0.828 

3.046 

4.85 

4.8 

13.524 

Total 

53.873 

54.825 

53.223 

59.981 

222.202 

All 

Services 

Total 

155.435 

139.130 

139.019 

147.279 

580.863 

®  As  this  paper  was  going  to  press,  we  were  informed  that  these  figures  reported  by  MATRIS  should  be 
corrected  as  shown  below.  Please  note  that  we  were  NOT  able  to  make  these  corrections  in  the  body  of 
this  report. 


Project 

FY92 

FY93 

FY94 

FY95 

Total 

1123 

6.139 

7.225 

5.856 

6.754 

25.974 

2743 

4.484 

4.981 

4.462 

4.448 

18.375 

Overhead 
(civilian  salaries, TOY,  O&M) 

2.058 

2.241 

1.744 

1.744 

7.757 

TOTAL 

12.651 

14.447 

12.062 

12.946 

52.106 
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Projects  and  Expenditures  for  R&D  on  Simulation  and  Training,  FY  1993, 

by  Service 


I.  Army 

1.  PE  0602727 A,  Project  A230:  Non-system  training  devices,  STRICOM 
Description:  DIS  improving  and  enabling  research. 

FY93:  $8,483  M 

2.  PE  06027 85 A,  Project  A790-HF:  Human  j)erformance  effectiveness  and  simulation, 
ARI 

Description:  how  to  design  simulators  that  fit  people. 

FY93:  $0.00 

3.  PE  0602785A,  Project  A790-ST:  Human  performance  effectiveness  and  simulation, 
ARI 

Description:  how  to  design  simulators  that  fit  people. 

FY93:  $0.00 

4.  PE  0602785A,  Project  A791-ET:  Education  and  training  technology,  ARI 
Description:  how  to  design  simulators  that  fit  people. 

FY93:  $4.273M 

5.  PE  06027 85 A,  Project  A791-MP:  Education  and  training  technology,  ARI 
Description:  research  on  best  training  methods  with  simulators 

FY93:  $4.806M 

6.  PE  0603007 A,  Project  A794:  Education  and  training,  ARI 
Description:  methods  for  training  enhancement 

FY93:  $4.992M 

7.  PE  0603003A,  Project  DB39:  Flight  Simulator  Components,  STRICOM 
Description:  DISATD 

FY93:  $2.561M 

8.  PE  06047 15 A,  Project  D241:  Simulation  and  Training  Devices,  STRICOM 
Description:  enhanced  technology  in  various  simulators/simulations. 

FY93:  $16.952M 

9.  PE  06047 15A,  Project  D396:  Tactical  Simulation,  STRICOM 
Description:  enhanced  simulations. 

FY93:  $2.547M 

10.  PE  06048 lOA,  Project  D275:  Synthetic  flight  training  systems,  STRICOM 
Description:  technology  enhancements  in  flight  training. 

FY93:  $6.336M 

11.  PE  06047 1 5 A,  Project  D574:  Combined  Arms  Tactical  Trainer,  STRICOM 
Description:  CCTT  acquisition  activity. 

FY93:  $14.617M 
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11.  Navy 

12.  PE  0602233N,  Project  RM33D40:  Tactical  Decision  Making  under  Stress 
(TADMUS),  NTSC 

Description:  human  performance  study. 

FY93:  $1.845M 

13.  PE  0602233N,  Project  RM33T24:  Simulation  T^hnology,  NTSC 
Description:  enhanced  technology  for  training  on  various  systems. 

FY93:  $1.249M 

14.  PE  0602233N,  Project  RM33T25:  Virtual  environment  training  technology,  NTSC 
Description:  improved  visual  technology. 

FY93:  $2.645M 

15.  PE  0603707N,  Project  L1733:  Simulation  and  training  devices,  NTSC 
Description:  various  technology  enhancements. 

FY93:  $5.280M 

16.  PE  0603792N,  Project  R1889:  Carrier  Based  Weapon  Systems  Trainer  (CV  WST), 
NTSC 

Description:  new  training  tool. 

FY93:  $2.500M 

17.  PE  06047 14N,  Project  W2 124:  Air  Warfare  Training  Development,  NADC 
Description:  new  software. 

FY93:  $2.1 19M 

18.  PE  0604715N,  Project  S1427:  Surface  Tactical  Team  Trainer,  NTSC 
Description:  soft  and  hardware  for  simulations. 

FY93:  $3.100M 


ni.  Air  Force 

19.  PE  0602205F,  Project  1 123:  Aircrew  Training  Technology,  AL/HRA 
Description:  methods  for  enhancd  training. 

FY93:  $8.71  IM 

20.  PE  0603227F,  Project  2743:  Tactical  Multi-ship  Aircrew  Training  Research,  AL/HRA 
Description:  find  limits  of  DIS  to  tactical  fighter  air  force. 

FY93:  $5.927M 

21.  PE  0604227F,  Project  3257:  Helmet-Mounted  Technology,  AL/HRA 
Description:  new  visu^  tool. 

FY93:  $4.5 17M 

22.  PE  0604227F,  Project  2325:  Simulator  Development  Activity,  Training  Systems 
Special  Projects  Office 

Etescription:  hard  and  software  developments. 

FY93:  $3.600M 

23.  PE  0604227F,  Project  2769:  Simulator  Update  Development/Simulator  Requirements 
Definition,  Training  Systems  Special  Projects  Office 

Description:  improvements  of  existing  simulators. 

FY93:  $8.525M 
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24.  PE  0604227F,  Project  2851:  Standard  DoD  simulator  database/common 
transformation  program,  Training  Sytems  Special  Project  Office 
Description:  software  standards. 

FY93:  $3.600M 

25.  PE  0604227F,  Project  2901:  B-IB  Weapon  Systems  Trainer,  Training  Systems 
Special  Projects  Office 

Description:  weapons  systems  trainer. 

FY93:  $3.700M 

26.  PE  0604227F,  Project  3000:  KC-135  Aircrew  training  system. 

Description:  weapons  systems  trainer. 

FY93:  $0,000 

27.  PE  0604227F,  Project  2968:  Modular  Simulator  Design,  Training  Systems  Special 
Projects  Office 

Description:  software  standardization. 

FY93:  $0.620M 

28.  PE  0604227F,  Project  3282:  C-17  Aircrew  Training  System,  Training  Systems 
Special  Projects  Office 

Description:  weapons  systems  trainer. 

FY93:  Sl.lOOM 

29.  PE0604227F,  Project  3722:  C-141  Aircrew  Training  System,  Training  Systems 
Special  Projects  Office 

Description:  weapons  systems  trainer. 

FY93:  $0.620M 

30.  PE  0604227F,  Project  4022:  Simulator  for  Electronic  Combat  Training  (SECT), 
Training  Systems  Special  Projects  Office 

Description:  weapons  systems  trainer. 

FY93:  $10.674M 

31.  PE0604227F,  Project  4033:  JP ATS  Training  System,  Training  Systems  Special 
Projects  Office 

Description:  weapons  systems  trainer. 

FY93:  $0,000 

32.  PE0604243F,  Project  3818:  Maintenance  Skills  Tutors  (MST),  AL/HRD 
Description:  weapons  systems  trainer. 

FY93:  $3.046M 
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